Test Management (system testing - automated)

Tools

 

Gui Testing (on Test Management)

Sources can be found here : https://scm.gforge.inria.fr/anonscm/svn/gazelle/Maven/gazelle-gui-testing/Automated-Test-testNG/trunk (website link)

This project, based on Automated Testlink scenarios project do the same job : automates scenarios written in Testlink for Gazelle-TM. All the test classes have been updated, the XPaths library has been also updated to be compliant with new Gazelle Test Management interface.

The Development environment and Maintenance sections are only about the Gui Testing project. 

Pages Testing (on Test Management)

Sources can be found here : https://scm.gforge.inria.fr/anonscm/svn/gazelle/Maven/gazelle-gui-testing/PagesTesting/branches/selenide_testing (website link)

This project, using Page Object like in Gui Testing project automates scenarios written in Testlink for Gazelle-TM. It is used to test the access rights on all pages for each existing user profile within Gazelle TM

Old Tools (no updated since 2014)

Menu checker

Sources can be found here : https://scm.gforge.inria.fr/svn/gazelle/Maven/gazelle-gui-testing/topmenu (website link)

This tool browses every menu item with every account and with every Gazelle mod (tm, gmm, etc.). The concerned menu is the main menu on the top of the page. The tool just visit the page and checks that it does not redirect to error.seam. Checks are performed on the title, the url of the page, or on the text contained in the page. Errors can be defined either directly in the class ErrorPages or in an external file errorPages.xml. Logs are put in the /log folder. They contains the list of errors found, a screenshot of before/after encoutering the error, and the pages error.seam themselves (debug info can be stored here). The tool can be run by calling mvn -test (which is useful for jenkins) or by building a jar. See the specific section for more details.

EJB dependency : an EJB is used by the menu checker to get the list of the menu items. This dependency is set in the pom.xml. It should be updated if the one used in the tested gazelle installation differs from the one set in pom.xml.

Monkey testing

Sources can be found here : https://scm.gforge.inria.fr/svn/gazelle/Maven/gazelle-gui-testing/monkey-testing (website link)

This tool works the same way as the menu checker, except it clicks on a random link in the page. A time is set for a given session in the config.properties file and the tool travels through gazelle for that given time, then switches to another profile, etc. Logs work the same way as menu checker. The latest version is still a bit old and can only be run with mvn -test.

Automated Testlink scenarios

Sources can be found here : https://scm.gforge.inria.fr/svn/gazelle/Maven/gazelle-gui-testing/Automated-Test-testNG (website link)

This project automates scenarios written in Testlink for Gazelle-TM. The purpose of this tool is to be run with Jenkins at a given rythme, to give a constant view of the state of Gazelle-TM. The can be run with mvn -test. It is possible to configure Jenkins in such a way that it logs all test results in Testlink. See the specific section for more details. It is possible to add more test pretty easily, this is explained here

Development environment

Dependencies

Several dependency are required for the project to work. They are all included in the pom.xml :

  • TestNG
  • Selenium (only version 2.X)
  • Gazelle-TM

TestNG eclipse plugin must be installed for test development. It can be found in the eclipse marketplace (TestNG for eclipse) or installed manually : Help > Install new software > fill in with "http://beust.com/eclipse". Further instruction on TestNG here.

Selenium and browsers configuration

Selenium requires a Mozilla Firefox installed in the developper's computer. To be able to run tests on Mozilla Firefox, you need 46.0.1 version or older, after 47, Selenium is not able to driver the browser.

NEVER UPDATE MOZILLA FIREFOX VERSION, WEBDRIVER IS NO MORE MANAGED BY SELENIUM BY UPDATE 47

The project allow you to choose if you want to use Mozilla Firefox browser or Google Chrome. To do this, you will need to install chromedriver binary on the developper's computer.

You can find what you want on this page. If you choose to use Google Chrome instead of Mozilla Firefox, just edit Maven's pom.xml browser property, on concerned profile. (firefox, chrome are accepted).

Test Development

When developping a test, it is possible to run it by pressing Shift + Alt + X then N (or by clicking Run > Run as > TestNG test) when you're using Eclipse. If you're using Intellij IDEA you can run the current test by pressing Shift + F10 (or by clicking Run > Run 'CLASS_NAME').

Make sure that the "headless" property in pom.xml current used profile is set to false, otherwise the test will be run with xvfb (which can be useful, though). It can run in debug mode, like any other program.

Maintenance

The issue that is most likely to happen is xPath deprecation. Some changes won't affect the ability of xpaths to select the right object, but some will. The most vulnerable xpaths are those with a "nth" constraint, as "//table/tr/td[2]". This one for example select the second cell of a line (on the second column). If a column is added at the begining of this table, the xpath will stop selecting what it should. There are two kind of xpath deprecation :

  1. The xpath selects nothing
  2. The xpath selects another object

The first case is not that hard to fix. Because all xpath are named and all their call encapsulated in "getWebElement()", all exception will be caught. If, for example, I use the xpath "Button.Login", that has deprecated because the webpage changed, the method getWebElement() will display a console message of this kind : "[Button.Login] can't be found. Called by [ThisPageObject]". In this case my xpath stopped working but I know exactly which one and in which page. Half of the job is already done, I just now have to find a new xpath for Button.Login.

The second case can be tricky. Suppose the xpath "Td.Name" select the first column of a table with two columns : name and age. Suppose now that a new column is added at the first position : Surname. The xpath will still select something without error, but it won't be what we want to select. No direct error message will be displayed. The most likely outcome is that an assertion will fail. It is possible that this problem will require some more debugging just to find the erroneous xpath.

Another kind of problem is a webpage whose behavior changes. In this case, not only the xpaths, but the PageObject and the test classes are deprecated and have to be remake.

The most dangerous change (by far) is an update of seam. The way seam converts java component into html/css/javascript ones is decisive. PageObject class relies a lot on how the page is implemented : if one xpath says "I want the link inside this cell of this table", it won't work if the link is placed inside a div. But that change could happen, seam wraps its components inside a lot of invisible containers (div, span). It is completely possible that, for example, seam changes the way it renders its table in html, and that all cells of tables (td) become wrapped inside a div. When this happen, a lot of tests will suffer. It is a risk and will maybe never happen (given the last update of the seam website, it is indeed no very likely) but it is important to know this weakness.