Sunday, June 29, 2008

Keyword or Table Driven Test Automation

Keyword or Table Driven Test Automation

Nearly everything discussed so far defining our ideal automation framework has been describing the best features of "keyword driven" test automation. Sometimes this is also called "table driven" test automation. It is typically an application-independent automation framework designed to process our tests. These tests are developed as data tables using a keyword vocabulary that is independent of the test automation tool used to execute them. This keyword vocabulary should also be suitable for manual testing, as you will soon see.

Action, Input Data, and Expected Result ALL in One Record:
The data table records contain the keywords that describe the actions we want to perform. They also provide any additional data needed as input to the application, and where appropriate, the benchmark information we use to verify the state of our components and the application in general.

For example, to verify the value of a user ID textbox on a login page, we might have a data table record as seen in Table 1:

WINDOW

COMPONENT

ACTION

EXPECTED

VALUE

LoginPage

UserIDTextbox

VerifyValue

"MyUserID"

Table 1

Reusable Code, Error Correction and Synchronization:
Application-independent component functions are developed that accept application-specific variable data. Once these component functions exist, they can be used on each and every application we choose to test with the framework.

Figure 2 presents pseudo-code that would interpret the data table record from Table 1 and Table 2. In our design, the primary loop reads a record from the data table, performs some high-level validation on it, sets focus on the proper object for the instruction, and then routes the complete record to the appropriate component function for full processing. The component function is responsible for determining what action is being requested, and to further route the record based on the action.


Framework Pseudo-Code

Primary Record Processor Module:

   Verify "LoginPage" Exists. (Attempt recovery if not)
   Set focus to "LoginPage".
   Verify "UserIDTextbox" Exists. (Attempt recovery if not)
   Find "Type" of component "UserIDTextbox". (It is a Textbox)
   Call the module that processes ALL Textbox components.

Textbox Component Module:

   Validate the action keyword "VerifyValue".
   Call the Textbox.VerifyValue function.

Textbox.VerifyValue Function:

   Get the text stored in the "UserIDTextbox" Textbox.
   Compare the retrieved text to "MyUserID".
   Record our success or failure.

Figure 2

Test Design for Man and Machine, With or Without the Application:
Table 2 reiterates the actual data table record run by the automation framework above:


WINDOW

COMPONENT

ACTION

EXPECTED

VALUE

LoginPage

UserIDTextbox

VerifyValue

"MyUserID"

Table 2

Note how the record uses a vocabulary that can be processed by both man and machine. With minimal training, a human tester can be made to understand the record instruction as deciphered in Figure 3:

On the LoginPage, in the UserIDTextbox,
Verify the Value is "MyUserID".

Figure 3

Once they learn or can reference this simple vocabulary, testers can start designing tests without knowing anything about the automation tool used to execute them.

Another advantage of the keyword driven approach is that testers can develop tests without a functioning application as long as preliminary requirements or designs can be determined. All the tester needs is a fairly reliable definition of what the interface and functional flow is expected to be like. From this they can write most, if not all, of the data table test records.

Sometimes it is hard to convince people that this advantage is realizable. Yet, take our login example from Table 2 and Figure 3. We do not need the application to construct any login tests. All we have to know is that we will have a login form of some kind that will accept a user ID, a password, and contain a button or two to submit or cancel the request. A quick discussion with development can confirm or modify our determinations. We can then complete the test table and move on to another.

We can develop other tests similarly for any part of the product we can receive or deduce reliable information. In fact, if in such a position, testers can actually help guide the development of the UI and flow, providing developers with upfront input on how users might expect the product to function. And since the test vocabulary we use is suitable for both manual and automated execution, designed testing can commence immediately once the application becomes available.

It is, perhaps, important to note that this does not suggest that these tests can be executed automatically as soon as the application becomes available. The test record in Table 2 may be perfectly understood and executable by a person, but the automation framework knows nothing about the objects in this record until we can provide that additional information. That is a separate piece of the framework we will learn about when we discuss application mapping.

Findings:
The keyword driven automation framework is initially the hardest and most time-consuming data driven approach to implement. After all, we are trying to fully insulate our tests from both the many failings of the automation tools, as well as changes to the application itself.

To accomplish this, we are essentially writing enhancements to many of the component functions already provided by the automation tool: such as error correction, prevention, and enhanced synchronization.

Fortunately, this heavy, initial investment is mostly a one-shot deal. Once in place, keyword driven automation is arguably the easiest of the data driven frameworks to maintain and perpetuate providing the greatest potential for long-term success.

Additionally, there may now be commercial products suitable for your needs to decrease, but not eliminate, much of the up-front technical burden of implementing such a framework. This was not the case just a few years ago.

Monday, June 23, 2008

Write to excel file

Found an interesting peice of code to write to xls.
To write directly to an Excel-file (with formatting) or to a
simple csv-file (without formatting).

To write the result of a test to an Excel-File, you can do the
following:

excel = WIN32OLE::new('excel.Application')

workbook = excel.Workbooks.Add

worksheet = workbook.Worksheets(1)

worksheet.SaveAs("test.xls")

#the headline

worksheet.range("a1").value = "Action"

worksheet.range("b1").value = "Result"

#put a "try-catch" around the test, writing either "ok" to a green
cell or the error-text to a red cell:

begin

            …do something…

assert($ie.contains_text("Test"), "ERROR 1!")

            worksheet.range("a1").value = "Action 1"

worksheet.range("b1").value = "ok"

            worksheet.range("b1").Interior['ColorIndex'] = 43 #green

rescue => e

            worksheet.range("b1").value = e.to_s

            worksheet.range("b1").Interior['ColorIndex'] = 46 #red

end

worksheet.columns("a:b").AutoFit

workbook.save

workbook.close

excel.Quit

Writing a csv-File is simpler but lacks the ability to use formatting:

logfile = open("test.csv", "a")

logfile.puts "Action;Result"

begin

            …do something…

assert($ie.contains_text("Test"), "ERROR 1!")

            logRow = "Action 1;"

logRow += "ok"

logfile.puts logRow

rescue => e

            logRow = "Action 1;"

logRow += e

logfile.puts logRow

end

Accessing tables inside table using xpath

When we have multiple tables inside tables then accessing it becomes tough. For ex. if the xpath for an element is like '/html/body/table/tbody/tr/td/table[2]/tbody/tr[5]/td[2]/table/tbody/tr/td' then we can get the element using Watir::IE.table(:index, 4)[1][1]

Saturday, June 21, 2008

Accessing iframes using Ruby

"Watir allows access to frame objects by identifying them by their name attribute. This is the form that Watir takes to access a frame:
   ie.frame("menu")

To access individual objects within that frame, you prefix the object using the Watir code above. If we had a hyperlink in the menu frame: Click Menu Item, we could click it like this:
   ie.frame("menu").link(:text, "Click Menu Item").click

See [installation dir]/watir_bonus/doc/watir_user_guide.html

Also one can access the iframe using the frame ID. for ex.
ie.frame(:id,'frameID')

Wednesday, June 18, 2008

Watir Script Examples

My evaluation of Watir as an automation tool for my website is complete. I really like this tool! It runs fast - a lot faster than Silk Test that I've used before. The only con right now is there's no record and playback (you have to look into WET or other Watir Maker). So far I haven't had too many issues finding the object on the page.

First start with the online tutorial and FAQ:

Watir works with IE 6 and IE 7, with the assistance of the IE Developer Toolbar. Navigate to your webpage with the toolbar. Create your Watir scripts using the Scite text editor that comes with the Ruby installation (colors make your life a whole lot easier). Save files with a .rb extension. To run, simply find the script.rb and double click it.

With an IE browser window open with the tool bar, click the arrow on the left and select the object on your page. You'll see the attribute - usually an id, name, href, and so on. You can just follow the basic syntax below (comments are followed by the # symbol):

browser.object(:attribute,"value").action
$ie.button(:id,"button1").click

Start of a script:
require 'watir'
$ie = Watir::IE.new
$ie.set_fast_speed #Compare this with $ie.set_slow_speed, you'll see a big difference!
$time = Time.now.strftime("%m/%d/%y-%H:%M:%S") # Get the time now and format to string. Can also try $time = Time.now.to_s
$ie.goto("www.newsqa.com") #Note that $ next to the variable name is not required.

-----------------------------------------------
Watir/Ruby code examples:

Read from a file in Watir script:
-----------------------------------------------
#Write to a file in Watir script: Open the file you want to write to in append mode. If a file exists, it'll append #to it. If not, it creates a new one

File.open('nameYourFile.txt', 'a') do |f|
f.puts "Writing to file. Code gets indented"
end #Make sure to end the loop
f.close # close the file when done writing to it
------------------------------------------------
Bypassing an Internet Explorer 7 security warning:
if $ie.text.include? "Login" #checks that text is included on the page
puts "Login page found, so login!" #puts writes to the command window
else
f.puts "Security Alert must be in the way. Click to bypass"
$ie.link(:id, "overridelink").click
f.puts "Clicked the continue link in IE 7"
end
------------------------------------------------
Bypass Internet Explorer 6 security warning:
------------------------------------------------
Bypassing a Javascript popup warning
These are pretty tricky. After much research, I got it to work by adding this to the script:

require 'watir'
require 'watir/winClicker'
require 'watir\contrib\enabled_popup'

def startClicker( button , waitTime= 9, user_input=nil )
#get a handle if one exists
hwnd = $ie.enabled_popup(waitTime)
if (hwnd)
w = WinClicker.new
puts ("yes there is a popup")
if ( user_input )
w.setTextValueForFileNameField( hwnd, "{user_input}" )
end

#"OK" or whatever the name on the button is
w.clickWindowsButton_hwnd( hwnd, "#{button}" )
puts ("Pop up button is: #{button}")
sleep 2
w=nil
end
end

# Use click_no_wait on the object that triggers the Javascript warning
$ie.link(:id, "deleteProduct").click_no_wait
#In the section of the script where the popup occurs:
startClicker( "OK", 3)

Tuesday, June 17, 2008

Watir 1.5.6 released with "Zippy" speed by Bret Pettichord

Found this article at http://www.io.com/~wazmo/blog/archives/2008_06.html#000279




Watir 1.5.6 released with "Zippy" speed

Last week we released a new version of Watir. We've mostly been accumulating bug fixes and minor enhancements. However there is one new feature that may be worth the upgrade. We've added a speed improvement to Watir, which we call "Zippy" speed.

Installation

Watir now has a two-step install process (assuming you've already installed Ruby). Type this at the command prompt:
gem update --system
gem install watir
This will update your version of the Rubygems installer and then install the latest version of Watir. Some people have had install problems with 1.5.4 and 1.5.5. Those problems are now fixed.

We are also now recommending that people use Ruby 1.8.6-26 final with Watir. A Ruby bug affecting many Watir users in earlier versions of 1.8.6 is now fixed. If you are upgrading your version of Ruby, you basically have to do a fresh install. There is no upgrade option. Sorry.

Zippy Speed

When you first use Watir, it is rather slow. This is intentional. We made Watir run slow at first because we found that new users wanted to be able to watch their scripts and make sure they were working correctly. So we added some delays to slow Watir down to an auditable speed.

However, once you have confidence in your tests, you probably want to run them faster. This is just one command.

ie = Watir::IE.new
ie.speed = :fast
I know this is a surprise to some Watir users: we haven't done a great job of letting this trick be known. This feature has been in Watir for a long time, certainly as far back as Watir 1.4.

What is new is that we now have an even faster way to run Watir. We call this "zippy" and it works the same way.

ie = Watir::IE.new
ie.speed = :zippy
Normally, when Watir enters text into a text field, it does so letter by letter. Two reasons. One, it looks cooler. Two, there are rare cases where you have to do this to realistically engage Javascript attached to the text field that, say, limits the characters that can be entered. But with tests that had lots of data entry, this could really slow things down.

The new :zippy speed speeds up tests by enter text fields all at once. If you have tests that require character-by-character text entry, you'll need to continue to use :fast.

In the past, many Watir users realized that they could speed up their tests by using value instead of set. Thus, instead of

ie.text_field(:id, 'payment').set '25'
They would use
ie.text_field(:id, 'payment').value = '25'
The new :zippy speed does exactly the same thing, only behind the scenes so you don't have to modify your individual scripts.

Zippy speed was originally included in Watir 1.5.4, but I've been reluctant to brag about it until we got our install problems fixed. It is based on a proof of concept by Jonathan Kohl.

Should You Upgrade?

Other features new since 1.5.3 (released last fall) include support for Chinese character input and the ability to locate more elements using multiple attributes. However, multiple attribute support for text fields, buttons and other input elements has yet to be implemented. We know you are waiting for it.

You can now call the row method from within a table, also frequently requested. There are a number of other fixes. (See the Release Notes for details.)

I know many of you are still using Watir 1.4.1. There is only one reason I know of why you should not upgrade. We have had several reports that Watir 1.5.3 and later is slower with some complex pages. We recently got a reproducible case of this problem and will be looking at fixing it in 1.5.7.

DllRegisterServer in AutoItX3.dll failed. Return code was: 0x80070005

Symptom

When we try to registerdll we will get the following error

---------------------------
RegSvr32
---------------------------
DllRegisterServer in AutoItX3.dll failed.
Return code was: 0x80070005
---------------------------
OK  
---------------------------


Solution

Return code was: 0x80070005

If you are logged on with a user account that does NOT have administrative privileges, you will experience this behavior if the DLL file writes to the registry, or changes files in the %SystemRoot%\System32 folder.

To workaround this behavior, log on with an account that has administrative privileges.


unknown OLE server: `AutoItX3.Control' (WIN32OLERuntimeError)

Symptom
Following error is displayed when we run the script
D:/Installations/Ruby/lib/ruby/gems/1.8/gems/watir-1.5.6/./watir.rb:129:in `initialize': unknown OLE server: `AutoItX3.Control' (WIN32OLERuntimeError)
    HRESULT error code:0x800401f3      Invalid class string      from D:/Installations/Ruby/lib/ruby/gems/1.8/gem
s/watir-1.5.6/./watir.rb:129:in `new'        from D:/Installations/Ruby/lib/ruby/gems/1.8/gems/watir-1.5.6/./watir.rb
:129:in `autoit'        from D:/Installations/Ruby/lib/ruby/gems/1.8/gems/watir-1.5.6/./watir/ie
.rb:425:in `autoit'        from D:/Installations/Ruby/lib/ruby/gems/1.8/gems/watir-1.5.6/./watir/ie
.rb:413:in `bring_to_front'        from D:/Installations/Ruby/Projects/Learning/Create_Company_Order_Servic
es/Verify patch number.rb:41:in `goto'        from D:/Installations/Ruby/Projects/Learning/Create_Company_Order_Servic
es/Verify patch number.rb:47
D:\Installations\Ruby\Projects\Learning\Create_Company_Order_Services>regsvr32 AutoItX3.dll


Solution
You need to register AutoIt. In a command window, navigate to the directory where you've put AutoIt and type:
regsvr32 AutoItX3.dll