We get the following three questions a lot from perspective users and customers. Here are my stock answers:

1. I am a bit confused about the license. Can the client use the GPL-d system without commercial license? Or is the commercial license mandatory? Also, is there a difference between the commercial and the GPL-d system?

PushToTest distributes pre-built and ready-to-run TestMaker under a simple free commercial license. The license limits you to run performance tests up to 200 concurrent simulated users. We sell licenses for additional simulated users.

PushToTest also distributes the source code to TestMaker under a GPL version 2 free open-source license. The code does not come with the installer and some other options. A comparison is found at http://www.pushtotest.com/Downloads/compare

Most of our users download the pre-built version. They don't have the technical skills necessary to compile, install, operate, and support a test platform. That's our value add to the software source code. And, we generally cost a fraction of the commercial test tools.

2. Do you know whether pushtotest has ever been used for WebMethods? If so, do you have a reference or evaluation or article? My client mainly uses the messaging capabilities of WebMethods (jdbc-adapter, MQ-adapter and trading networks).

We are partners with Software AG. We provide a Software As A Service (SAAS) interface between the WebMethods X-Registry. We also use TestMaker to test ESB and BPM products for scalability and performance. See http://soakit.pushtotest.com for details.

3. Is the monitoring tool part of PushToTest or do you use something like Glassbox?

Monitoring is part of PushToTest TestMaker. Our monitors observe CPU, Network, and Memory utilization and correlated the findings back to the test operation automatically. Plus, we are working on Glassbox integration. See http://bugs.pushtotest.com/ticket/61. We also integrate with DBA InfoPower's database root cause analysis and mitigation service.

We will be renovating the Web site shortly to address these questions.

-Frank

Ole Matzura started a blog on soapUI topics. Ole is an excellent writer and hugely enthusiastic for using soapUI to test SOAP and REST-based Web services and Service Oriented Architecture (SOA.) The combination makes the blog a fun and informative read. The blog entries are like mini-tutorials. They feature screen shots and how-to step-by-step instructions. Check it out.

-Frank

I've been doing some thinking lately about how we build and test applications. I'm no agile expert but I am impressed with the concepts of test-first and iterative development. Waterfall seems like it is designed to kill developers, testers, and projects by its nature.

I've never really understood the test-first thing in its entirety. In an SOA environment, it seems to me that test-first would put the emphasis on defining interfaces first and building tests against the interface (both good things.) The interface is just an interface. So I wonder if having some of the application logic implemented behind the interface is required to really do test-first right.

It seems to me that some sort of enhancement of agile could help here. One where I build the application and the test at the same time and iteratively.

I am trying this out using an open-source Ajax development framework (Appcelerator) and my test framework (TestMaker.) This is just a bunch of notes right now. I'm hoping to have something coded in the next 2 weeks. Here's how it would work:

Appcelerator uses non-standard tags in HTML to implement Ajax functions. For example:

<script src="javascripts/appcelerator.js"
 type="text/javascript"></script>

This provides Appcelerator's function library to a page.

<input type="text" id="username" validator="required" decorator="required" fieldset="login"/>

When the page loads Appcelerator provides Ajax functions for validator, decorator, and fieldset attributes of the <input> tag.

<input type="button" value="login" activators="username,password" id="button"
on="click then r:login.request" fieldset="login"/>

The fieldset attribute is used to link <input> elements (or divs, spans) for the purpose of sending a message. The example above links the username to the submit button. When the user clicks submit the "on" attribute tells Appcelerator to click the submit button and send a login.request message to the server handling the log-in. (Appcelerator implements the server side component and the messaging for you.)

This kind of tag framework approach to application composition and development is wide spread and makes application development easier. In the above example, I didn't have to do any JavaScript work to deliver an Ajax enabled log-in page.

My thought is to extend the tags to include test definition tags. For instance:

<script src="javascripts/testmaker.js" type="text/javascript"></script>

<input type="text" id="username" validator="required" decorator="required" fieldset="login" test_input_data="dpl:mycsvfile field:username"/>

<input type="button" value="login" activators="username,password" id="button" on="click then r:login.request" fieldset="login"
test_verify="verifyTitle:Main menu"
/>

In the example above, when the page loads the testmaker.js functions respond to the test_input_data tag by reading from a comma-separated-value (csv) file using a Data Production Library (DPL.) The first row of the csv file defines field titles. TestMaker sets the username input field with the username value in the csv file.

When I want to test the page I use a test runner that loads the page and clicks the login button. The response to the login must be a page with a title of "Main menu". Otherwise, the test throws an exception and I could that as a failed functional test.

What I'm after here is a way to build the tests as we build the application.

I would love your feedback, thoughts, and ideas on this.

-Frank

Brian Gentile, CEO at Jaspersoft, told me about a Jasper meet-up next week. Here are the details. -Frank

---

Come join us at the next Bay Area Jasper Directions Meet-Up in Palo Alto.

Jaspersoft has been working hard to deliver the best in open source business intelligence.  Recently we announced the release of Jaspersoft v3, which brings Web 2.0 functionality to business intelligence.  We’ve also upgraded and enhanced JasperForge.org to make it more interactive and feature rich.

Learn more about these recent developments and what’s coming in the future at our upcoming Jasper Directions Fall 2008 Meet- up on Wednesday, October 1st at SAP Ventures in Palo Alto, California.

Jasper Directions will be hosted by three Jaspersoft luminaries: Teodor Danciu, founder of JasperReports, Giulio Toffoli, founder of iReport, and Sherman Wood, Director of Business Intelligence and Mondrian committer.

SAP Ventures- Bldg. D
Executive Briefing Center - Southern Cross Room
3410 Hillview Avenue Palo Alto, California
October 1, 2008
6:00 pm-8:00 pm

Organizations with investments in HP/Mercury solutions often find that they are unable to achieve agility, efficiency, and economy when confronted with test, monitoring, and operational management requirements in Service Oriented Architecture (SOA,) Rich Internet Applications (RIA) using Ajax, and Web services. PushToTest delivers solutions for organizations to extend the value of their HP/Mercury solutions and increase their existing skilled teams by adding fast, fun, easy, and inexpensive open-source test tools that leverage and extend HP/Mercury solutions.

Read about the PushToTest solution for HP/Mercury customers

Jeremy Geelen of SysCon TV interviewed me at the JavaOne 2008 conference. A streaming video of the interview is at: http://tv.sys-con.com/node/674567.

In the interview we talk about moves in the Open Source Test (OST) world to build a competitive test and operational management product stack. The stack provides record/playback, test scripting, functional testing, load and performance testing, and business service monitoring. The advantage of this stack is that it works in all environments: developers use it to turn unit tests into functional tests, testers use it to turn functional tests into load and performance tests, and IT managers turn functional tests into business service monitors. All in the same product!

I will be in Los Angeles/Anaheim next week to teach the Open Source Test Workshop. Tuesday, September 30, 2008 from 8 am to 12 pm. This is a free event! Here are the details:

Software developers, QA testers, and IT managers are challenged to rapidly build and test Ajax, Service Oriented Architecture (SOA,) and Rich Internet Applications (RIA) in a time when schedules are short, budgets are tight, standards are few, and much of this is new technology! Find out why AMD, Ford, and The Jackson Laboratory succeed with open source build and test software as a more affordable and flexible option to the traditional test vendors.

Learn how you can leverage open-source RIA application server technology from Appcelerator to rapidly build Ajax applications. Learn how you can leverage open-source testing tools (Selenium, soapUI, PushToTest, TestGen4Web, HTMLUnit) for functional testing, load and performance testing, and business service monitoring, with more flexibility than traditional solutions provide.

Here is the agenda:

Module 1: The why, what, and how of open-source build and test tools vs. traditional tools
Module 2: Building applications with Rich Internet Application technology without learning JavaScript
Module 3: Testing using record/playback and scripting tools
Module 4: Identifying and solving performance bottlenecks in Rich Internet Applications (Web 2.0 and Ajax)
Module 5: Service Oriented Architecture (SOA) testing
Questions and Answers

Register Now To Guarantee Your Seat At The Workshop

Six Questions to Consider Before Building a SOA Testing Team

SOA Expert Mike Kavis interviewed Randy Rice, Jim Giddings, and me for an article in CIO Magazine. The article discusses the challenges in testing and operational management of Service Oriented Architecture (SOA.)

Here is an excerpt:

What is the biggest change that SOA brings to testing?

RR: Randy states that "accessibility to services" or the "headless nature of services" creates new challenges for testers. Testers will have to test many services without the aid of user interfaces and must perform rigorous testing to validate these services. Randy says that testing services reminds him of the challenges of testing drivers which require test harnesses and testing frameworks.

FC: Frank talks about how services are "always available" which is different from n-tier architectures where "you are in control". With SOA you have services calling other services and in the world of mashups, testers do not always have knowledge of how these services will be used before deployment. Scale and performance are also high on Frank's list of challenges.

JG: Jim says you should expect frequent testing cycles much like agile projects and a test driven development methodology should be employed. Jim states that testing governance becomes critical in an SOA. Like his peers, Jim discusses the challenges of the headless nature of services and performance testing.

Read the rest of the article.

A business partner asked us recently: Where is PushToTest's primary audience? And, what are their job functions? We use Google Analytics to track our Web site activity. Google reports the following for August 16 to September 15:

United States, 3,315, 36.39%
India, 1,033, 11.34%
United Kingdom, 492, 5.40%
Germany, 386, 4.24%
Canada, 293, 3.22%
France, 260, 2.85%
Brazil, 204, 2.24%
Australia, 175, 1.92%
Costa Rica, 170, 1.87%
Italy, 166, 1.82%

I wish someone would invite me down to India for a couple of weeks to do a consulting project, teach the PushToTest methodology, and let me see the country.

I did a quick query of our registration list. Of the people that describe their job function:

      654  Test\QA Engineer                                   
      403  Developer                                          
      257  Project\Program Manager                            
      256  Application\IT Architect                           
      201  QA Manager                                         
      166  Consultant                                         
      121  Student\Faculty                                    
      112  Performance Engineer                               
       98  Director\VP - Application Development\IT           
       98  Business\IT Analyst                                
       94  Partner                                            
       86  Executive\C-Level                                  
       85  Portfolio Manager                                  
       81  Process Manager                                    
       76  Director\VP - Quality Assurance                    
       76  Other                                              
       75  Product Manager                                    
       64  Director\VP - Project Management Office            
       50  VAR\OEM Partner                                    

When I look at page view activity I see an equal number of software developers, QA testers, and IT managers. Earlier this year I spoke at the Selenium User meeting. I asked for a show of hands on the questions:

How many of you are software developers that also test?
How many of you are testers that also write software?

90% of the attendees answered 'yes' to the above questions. Maybe the message we've been yelling for so long is taking effect! And it a very good way!

-Frank

Many developers at PushToTest read and participate in the InfoQ on-line community. QCon is an excellent  conference on enterprise software development hosted by InfoQ, JAOO, and sponsored by PushToTest.

QCon San Francisco 2008 is coming up on November 19-21, 2008. The conference features 80 technical sessions (Nov 19-21) and 2 days of tutorials (Nov 17-18.)

We arranged a special discount for PushToTest readers of an additional $40 off. Use the discount code "pushtotest_qconsf2008" when you register for the conf.

http://qconsf.com/sf2008/conference


-Frank

We're packaging a minor bug fix and feature enhancement release. This will be TestMaker 5.2.1.

The following is a list of changes:

#289 Selenium waitForCondition support
#265 Data-driven Tests Need hasMore() function
#272 Installer does not install Monitor For windows (0 bytes)
#295 TestMaker must not erase TestScenario comments
#296 Selenium RC not working with Firefox 3
#237 PTTMonitor throws NPE on second call to stop function
#239 Exception trace prevents Summary report

Changes are found at http://bugs.pushtotest.com.

Thanks for all those that provided suggestions, patches, and feedback!

-Frank

We are building Selenium support into PushToTest TestMaker 5.2. The idea is to support Selenium IDE in Firefox for authoring and replay of tests in the browser. TestMaker will support Selenium RC to test Web applications across browser and operating environment versions. TestMaker will reuse Selenese tests in TestMaker's distributed test environment using HTMLUnit and Rhino as load and performance tests and business service monitors. Selenium tests have full access to operational test data using TestMaker's Data Production Library (DPL) system. We posted details at http://selenium.pushtotest.com.

For Selenium users this means they have a new option for Selenium training and support. PushToTest offers Selenium training courses and professional support agreements. The move gives organizations new uses for their Selenium tests as load and performance tests and service monitors.

The new Selenium IDE and HTMLUnit/Rhino support is already in the code base and will appear in TestMaker 5.2 beta 6 next week. The code is already in svn at http://dev.pushtotest.com/svn/tm5.

We added Selenium RC too. Tests may run in our TestNetwork grid in a similar fashion to Selenium Grid.

We would appreciate any feedback on our approach.

Frank Cohen
CEO and Founder
PushToTest

A bunch of small things today:

We're making good progress towards shipping TestMaker 5.2. Click here to view the remaining change list.

Building Selenium support for testing Ajax has taken us the past 4-5 weeks. We've found a number of problems and solved most of them. We found problems in the SeleniumHTMLUnit, HTMLUnit, and NekoHTML. We will contribute back all of the changes back to their respective projects.

Jeff Haynie of Appcelerator is extrodinarily fast at finding JavaScript and HTMLUnit problems!

Our engineering community are "heads-down" to get a 5.2 release out by Friday August 22, 2008.

We switched to svn: http://dev.pushtotest.com/svn/tm5

We still need to add a solution to the link="locator" library for SeleniumHTMLUnit.

TestMaker 5.2 will also support Selenium RC!

Watching the kids return to school and the Summer slowly slipping into Fall. Got my first issue of Powder magazine

-Frank

I will be speaking at STARWest, the software test automation conference, and PushToTest will be exhibiting at the conference. STARWest runs September 30 to October 3, 2008 at the Disneyland Hotel in Anaheim, California. PushToTest will be in booth 44.

I will be giving a talk on Open Source Test titled "Getting Open Source Test Tools to Work Together". Budgets are crunched and yet we still need to test. Open Source Test (OST) tools make test creation easy and efficient. Web, SOA, SOAP, and Ajax protocols are on the move and these tools provide much needed support testers.

We're planning to give away a new open-source test platform on a shiny new memory stick (pcmcia). I would be glad to send you one in the mail if you cannot attend the conference. Send me an email at frank@pushtotest.com with your address. I'll do my best to send these outside of the USA.

-Frank