Commercial Tools converging to Open Source counterparts!

SmartBear recently announced the launch of TestComplete 10.5 and QAComplete 9.9.5. The major change, to look for, in this new parcel is the support of Selenium WebDriver, the most used open source web testing automation tool. 

The way Selenium has appealed testers worldwide has attracted SmartBear's attention and they want Selenium's existing users to be able to switch their existing scripts to TC, without much of effort. Well, how much of it is true, time will only tell.

It's obvious that SmartBear wants to extend their user base and this move will actually help them achieve the goal but what impact will it make for the Selenium users? Will they be really interested in moving from totally open source eco-system to a paid one. Well, it again depends on what SmartBear offers to test teams through TC. They believe that Selenium lacks certain qualities like test case tracking, reporting, pass fail analysis, etc. Through this integration, SmartBear claims to provide testers use most of existing features of TC on existing scripts of Selenium.

On a high level they claim:

• Ability to debug failed Selenium scripts quickly using screenshots.
• In-depth reports allow testers to get more insights into code coverage and quality
• Perform test case prioritization by checking for traceability between requirements, manual test cases, automated tests including Selenium, and defects.
• Spend less time creating and maintaining Selenium scripts by reusing existing Selenium tests

The power of open source tools, in this case Selenium, can not be ignored. It's usage is very heavy in the industry because of ZERO licence cost involved (for management) and it's rich library of functions (for techies). But again, it fails to satisfy end-to-end needs of test teams. For example, Selenium doesn't offer in-depth reports and thus teams have to look for integrations with other 3rd party tools. SmartBear through this support claims to fill in that gap for all teams. 

Let's hope that TC actually fills this void through this integration, thereby, making difference to the lives of real testers who face various issues due to non-availability of certain rich features in Selenium. 

Which tests to Automate?

• Repetitive tests that run for multiple builds.
• Tests that tend to cause human error.
• Tests that require multiple data sets.
• Frequently used functionality that introduces high risk conditions.
• Tests that are impossible/tough to perform manually.
• Tests that run on several different hardware or software platforms and configurations.
• Tests that take a lot of effort and time when manual testing.

Automation Testing – Myth or Reality?

“Test Automation” is really misunderstood by many. With the intent to produce high quality software with ever-more complex technology under increasing competitive pressure, automated testing is being heavily adopted. This heavy adoption, over a period of time, have led many of us believe that it is a replacement of so-called “manual” testing.

Talking to one of the test managers about how he sees automation testing: “It’s good for everyone. Every client wants it and thus it’s important. Also, once you’ve the tests ready, the tester headcount may go down (just few are needed to execute tests). Best of all, automation results in huge ROI compared to manual testing.” This conversation actually shook me and led me talk to few other test managers to verify if that was just an individual’s opinion. But the truth was bitterer. The more I talked, the more shock I received. Most of them were having the same views as stated above.

This post is an attempt to spread the actual meaning of automated testing or to be precise automated checking and how it must coexist with manual testing.

Does Automated Testing really exist?

First of all, there’s nothing called as automated testing. It may sound, at first, little shocking but continue reading and I will prove it. We think of automation testing as: “a form of testing that utilises scripts to automatically run a set of procedures on the software under test, to check that the steps which are coded in the script work.” For example: if we had a script that logged into the website, then added an item to the cart and placed the order, a basic automated test would check that this path through the system is operational, which is an affirmation that the function operates without causing any known validation errors or exceptions and produces an expected output. But it does not check anything that is not written in the script. Does it?

The key word here, relating to automation, is “check”. As computers can’t think for themselves, they can only follow a set of commands that we give them, which offer a “yes/no” response. Anything that has a set “expected result” can be classed as a check, where no real sapience is required. (Michael Bolton has a great blog post on “Testing vs Checking” that is well worth a read if you haven’t already).

So, let’s start calling it as “Automated Checking” or just “Automation”, thereby making sense to what actually happens.

Automation is meant to replace Manual?

James Bach has very clearly explained it as: Contrary to the implication of typical marketing literature for test tools, automated testing is not the same as manual testing. What observant humans do when they go through a test process is in no way duplicated or replaced by test automation, because automation cannot be aware of all the hundreds of different failure modes that a human can spot easily. I have to explicitly program automation to look for suspicious flickers and performance problems, but with humans I can say "be alert for anything strange." So, no matter how many smart tools are available in market, there are still many things you can’t do with automation. Said this, it does not, in any sense, degrades the value of automation.

So, there will always be a need for manual, sapient testing in the software industry, but being able to utilise automation for the checking activities is highly beneficial. In order to have an effective process that focuses on building quality products in a fast paced environment, both testing methods are important for being able to achieve this! You need to utilise the benefits of having automation, in the fact that it will free up time and effort that a tester would normally have to spend performing the “checking” tasks, and allows the testers to focus more on the sapient testing that is required to be done in order to be able to discover an accurate level of quality that the software bestows.

Read more about Testing & Checking here.

JMETER - Part 1

JMeter is software that allows us to run load test or performance oriented business (functional) test on different protocols or technologies.

This is a Java desktop application with a graphical interface using the Swing graphical API, can therefore run on any environment / workstation accepting a Java virtual machine, for example: Windows, Linux, Mac, etc.

The protocols supported by JMeter are:

• Web: HTTP, HTTPS sites 'web 1.0' web 2.0 (Ajax, flex and flex-ws-amf)
• Web Services: SOAP / XML-RPC
• Database via JDBC drivers
• Directory: LDAP
• Messaging Oriented service via JMS
• Service: POP3, IMAP, SMTP
• FTP Service

JMeter Features

• It’s free & open source software.
• It has simple and intuitive GUI.
• JMeter can load and performance test many different server types: Web - HTTP, HTTPS, SOAP, Database via JDBC, LDAP, JMS, Mail - POP3
• It is platform-independent tool. On Linux/Unix, JMeter can be invoked by clicking on JMeter shell script. On Windows it can be invoked by starting the jmeter.bat file.
• It has full Swing and lightweight component support (precompiled JAR uses packages javax.swing.*).
• JMeter store its test plans in XML format. This means you can generate a test plan using a text editor.
• Its full multi-threading framework allows concurrent sampling by many threads and simultaneous sampling of different functions by separate thread groups.
• It is highly extensible.
• Can also be used to perform automated and functional testing of your application.

How JMeter Works?

JMeter simulates a group of users sending requests to a target server, and return statistics that show the performance/functionality of the target server / application via tables, graphs etc. The figure below depicts this process:

The Downside to Being a Tester!

The world is moving fast towards realizing the true importance of software testing. Gone are the days where management was biased against testers or testing teams. Today, we, the testers, are being looked upon as an integral part of product development. Well, the previous two statements still don't stand true for most of us, because most of us find ourselves working with not-so-great companies. With changing times, the expectations from testing teams have also been raised. The testers are now expected to test more complex scenarios with the intent to uncover more bugs than before. The world of testing is changing fast and the speed of change is no less for pure testers. By "pure" I mean "People who don't do anything else." While this may seem like a great career, and a cool way to get a first job in the business, the truth is less alluring.

Click Here for full article!

Password Recovery never's been so easy

There is a nifty utility allows you to reclaim your password: Activate the utility, drag the mouse cursor onto the password field and your password is revealed instantly.

Windows systems allow a convenient storage of frequently used passwords, such as the password of your dial-in ISP or FTP connection or GTALK or any other application. However, since you no longer enter the saved password manually, you tend to forget them. What are you to do when you need to know one of the saved passwords? The password is staring right at you, but is hiding behind a row of ***** asterisks.

For 9x and NT users the solution was to use a password recovery utility that displayed the passwords inside the box. But as many users upgrade to Windows XP they realize that Microsoft changed the security of the password boxes and all old password recovery tools fail on Windows XP. But with the new iOpus Password Recovery users can reveal their forgotten passwords on XP and Windows 2000 as easily as on Windows 98 or Windows ME.

iOpus Password Recovery XP is the first tool of its kind that works automatically on Web pages, Windows 9x/NT and Windows 2000/XP systems

Limitations:
Trial version will only display the 1st three letters of the password.

Workaround:
If you want to recover full password, without even paying for pro version, then paste full password and recover first three characters. Then paste it again, DEL DEL DEL i.e. delete first three characters, and doing this you recover next 3 characters of your password. Keep repeating this process until you get sufficient characters to remember your password.

Thanks to Santosh Tuppad for pointing this out.

The HALF-BAKED truth about Open Source Testing Tools

Open Source!!! Free!!! Sounds exciting! Isn’t it? Well, I am not against using open source tools and hence I’m going to say YES along with many of you. But everything has some limitations and open source testing tools are no exception. It is upto the users, aka, “the testers” to understand whether open source testing tools will be a good choice or not? Let’s enlighten ourselves with some of the limitations that are a part and parcel of most open source tools especially the ones for testers.

Read the complete article here!

Test Practitioner's Club - Mobile Apps Testing Meetup

Theme - Mobile Apps Testing

Time : 2:00 pm to 5:00 pm
Date : 29-Mar-2014

Agenda:

Session 1 : The Essentials of Mobile apps testing – 2:05 to 3:00PM
Speaker 1 : Bharath Nidikonda (Expedia)

Session 2 : Evolution of Mobile Test Automation Tools: Current & Future Trends – 3:05 to 4:00PM
Speaker 2 : Amitabh Srivastava (Impetus)

Session 3 : Android Automation using Calabash – 4:15 to 5:10PM
Speaker 3 : Kapil Saxena & Vaibhav Kushwaha (Magic Software)

Here are some pics:

Disable “Open File Security Warning” on WINDOWS

How to eliminate the “Open File Security Warning” from programs accessed from the file server?

1. Open the Control Panel
2. Open Internet Options
3. Click the Security Tab
4. Click on Local Intranet
5. Click on Sties
6. Click Advanced
7. Type the drive letter of your file server where the application is located in the “Add this website to this zone” box. 
8. Click Add
9. Click Close
10. Click OK
11. Close Internet Options by clicking OK
12. Close the Control Panel

 You should no longer get the “Open File Security Warning” when you run an application from the file server.

A/B Testing

A/B testing from Kapil Saxena

Database Checker

Database Checker helps you synchronize databases' structure changes between two MySQL databases, find broken foreign keys and many more. Database Checker is an OpenSource software, released under GPL licence, and free of charge.

Requirements

• PHP 5+
• MySQL 5+

(This may work for older versions, but not tested so far)

Download
Database Checker is provided without any warranty, use it at your own risk! If you don't know SQL language, you will lose data. Now that you're warned, here's the link: download Database Checker

Features
This tool was developed to see what where the changes between our development servers and our production servers. That's the main use of this tool. We added other things, such as table data changes (only on small tables, this is not a replication tool!).

Best is to start reading the Use cases/Reference to learn about what you can do with Database Checker.

1st meetup of Test Practitioner's Club

Venue:  Mohta Hall, PHD House, Near Siri Fort Auditorium, 4/2 Institutional Area, New Delhi.

Date:    1st Feb 2014

Time:    2 pm to 6 pm

Is there a such thing as manual testing?

Manual Testing!! Hmmm!

Wikipedia defines it as "Manual testing is the process of manually testing software for defects."

What does it actually mean? Why do we call it Manual Testing? Is it correct to call it Manual Testing? Aren't we just giving functional testing another name? Isn't it better to call it Functional Testing or simply Testing, rather than inappropriately calling it Manual Testing?

There's a need to understand the intent behind such naming convention. I am not 100% sure about it but my best guess is if the only intent was referring to fact that it's done by testers aka persons aka humans, then please for GOD's sake understand this: DEVELOPMENT is also done by humans but you don't call it "MANUAL DEVELOPMENT".

So why this biasing for testing?

Let's understand this basic thing and start calling ourselves as "testers" rather than "Manual Testers". Because for me there's no such thing as Manual Testers!

Can’t Identify an Object on Run Time?

During Test Automation, we identify various objects successfully, but many a times during script execution we face issues where our scripts fail to identify objects. What is the probable cause of such errors? Well, I'd say the test environment plays an important role in which you run your scripts.

Load relevant Add-ins

Make sure you have loaded ALL the relevant add-ins required for your Application under Test (AUT).

Disable Protected Mode

In IE, clear the Enable Protected Mode option under Tools > Internet Options > Security

User Account Control (UAC)

UAC was introduced in Windows Vista and continued in Win 7 and Win 8/8.1. UAC can interfere with your application. It is a good idea to disable it while working with any Test Automation tool.
• To turn-off UAC in Windows 7, go to Start Menu > Search for “Change User Account Settings” > Disable UAC
• To turn-off UAC in Windows 8/8.1, you need to take help of registry. UI setting doesn’t trulyturns-off UAC in Win 8/8.1. Hit Win + R key, type regedit and click OK. Navigate toHKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\Policies\System
Find the key: EbableLUA and change the Value Data value to 0

Zoom Level

Make sure the zoom level is set to 100% while testing in any browser. Check the illustration below on how zoom-level affects highlight object functionality in QTP. (Focus on the blinking black rectangle on the left once the Highlight button is clicked on the right.)

A shortcut key to get back to 100% zoom is Ctrl + 0 (that’s zero).

.Object Notation

There are times when identification properties are not enough to identify an object. You may make use of native properties in that case. Use .object notation to locate the erring property. Make sure to correctly identify it.

Low level recording

After you have exhausted all options you can try Low level recording. It’s not a good method from maintenance point of view but it may come handy at times.

My tool has worked for ages identifying objects correctly, stopped working since yesterday

If something like that has happened probably there is some change in your browser or UFT software itself. Try to disable auto-updates on machines where tool is installed.

Type property for WebButton class

If you’re testing cross browser, don’t use Type property to identify Webbutton class since the default value is different for IE (button) and Firefox (Submit).

Object with dynamic properties

If automation test tool was successfully able to identify object at record time but unable to identify the same object during replay time, there are chances that object properties are dynamic in nature. Make use of regular expression or parameterization to handle those dynamic values.

Version of Browser and Windows OS

Make sure the version of tool you use supports the version of your browser and/or your Windows operating system. Check the complete UFT vs. browser support matrix and UFT vs. Windows support matrix.

Install relevant patch

In case your application version is not supported by the version of tool, keep an eye on various patches that tool vendor comes up with from time to time. For example,

Forums

If nothing works for you, you can ask your question at various forums dedicated to the usage of tools. Please make sure your question is detailed enough to solicit good responses.

Testing XPath in Google Chrome browser

Many times during Test Automation, we use XPath to match the required object/tag in the DOM. While working with XPath, I'm wondering is there anyway to test XPath in Google Chrome browser with highlighting the matched tag in browser. The answer is YES. We can test XPath in Google Chrome.

Follow these steps to test your xpath:

1. Open Developer Tools - When you're in Chrome (on the page where the object is appearing), press F12

2. Select Console tab - just press ESC key and you'll see extra pane appearing at bottom. This is where you can type commands to test XPath.

3. Use $x token. For example, enter following in the console windows $x("/html/body") and then press ENTER. It will select the body tag and the result tag will appear just below command you entered.

In case, the entered XPath is wrong, it won't return anything.

Bingo!


NOTE: Unlike Firefox, Chrome has an in-built "developer-tools" option!

Defect Density!

Defect Density is the number of confirmed defects detected in software/component during a defined period of development/operation divided by the size of the software/component.

The ‘defects’ are:

• confirmed and agreed upon (not just reported).
• Dropped defects are not counted.

The period might be for one of the following:

• for a duration (say, the first month, the quarter, or the year).
• for each phase of the software life cycle.
• for the whole of the software life cycle.

The size is measured in one of the following:

• Function Points (FP)
• Source Lines of Code

DEFECT DENSITY FORMULA

USES

For comparing the relative number of defects in various software components so that high-risk components can be identified and resources focused towards them.

For comparing software/products so that quality of each software/product can be quantified and resources focused towards those with low quality.