A Beginner's Guide to Successful Testing at the Connectathon
This page is dedicated to the newbies but most of the tips apply to everybody so do not hesitate to go through this page. Might you have some other good tips to share, contact Anne-Gaëlle Bergé.
Many thanks to our contributors:
- Dave Franken
- Eric Poiseau
Help Each Other
This is so important that it's basically the Golden Rule. We're all in it together and to be successful with our tests, we need each other. Luckily, the Connectathon-floor is mostly "manager-and sales rep"-free, there are only helpful developers and there is a big camaraderie. Competition stops at the door. Trying to be helpful is great, but you also have to be honest. If someone asks you to do a test and you really don't have time for it, just say no. Don't make someone else wait for you and waste his precious time.
Read the Specs Carefully and Closely
Everything starts at the specs (Technical Framework). Make sure your implementation is based on the most recent version, there's a new release about every year. Supplements are released even more often, so be sure to pay close attention to when new versions are released. The specs are carefully constructed by committees, so usually there is little room for interpretation. Most disputes at Connectathon (you're doing it wrong! no, you're wrong!) are solved by just reading the specs carefully.
You would also follow the tutorials/trainings (http://gazelle.ihe.net/training). Although we try to make testing as easy as possible, this is not an easy task and discovering how the test tools are working on the connectathon floor is not a solution !
Test your implementations before coming to the Connectathon
This should be pretty obvious, but what I'm trying to say is that when you're planning to (re)develop an implementation for the Connectathon, you should leave plenty of room to test your software before actually going. The Connectathon shouldn't be used to test and redevelop your software (although that's inevitable), but mainly to get green check marks for software which already works. TLS/SSL is a big hurdle for most vendors, you really want to make sure this works before coming, otherwise you'll likely be spending and wasting the entire first day just to get SSL working.
Pre-Connectathon testing should guide you in the testing you need to perform before arriving on the connectathon floor.
Make sure your software is flexible
One mistake we've made in the past is that every time we wanted our consumer to connect to another registry, we had to recompile and redeploy our software with a new configuration. If you make sure you can easily connect your software to another system than the one that's currently configured, you will save a lot of time. You should only recompile and redeploy your software when you've made an actual change to the source due to a bug you've discovered, not because you want to connect differently.
Peers configuration can be imported from Gazelle Test Management. Gazelle allows you to export peers configuration in a CSV file that can then be easily used to feed your own application, avoiding you to enter them manually !
Make sure you can easily locate the log details a test needs
Most tests require you to upload log files or copy-paste some info from them. You should make sure that you can easily locate whatever the test needs. A mistake we've made in the past was to have every log info dumped to a single file which rotated every 2MB. It was a mess and we wasted a lot of time just to find what we were looking for. An idea here is to have a log per endpoint you're connected with or per transaction.
Using the proxy is one of the solution to make sure the exchanged messages are stored correctly. We will still do not have the ability to capture the logs of transaction under TLS, but for all the rest this is allow you easily capture the actual messages exchanged with the peers.
Manage your logistics
A lot of time at the Connectathon is spent on managing logistics, rather than actual testing. By this I mean finding partners to test with and streamlining tests. We always print out business card sized notes with our details (endpoints, table # and things like that). Skype is the de facto communication tool used at the Connectathon so you should have an account and add your test partners. This saves you from having to walk across the room every time you want to discuss something with your test partner. At Gazelle you can find everyone's system configs, so make sure you preload that information in advance.
Gazelle allows you to share a skype account information for each systems that you have registered. We recommend to use a dedicated account, one you create for the purpose of the event. Note that all your peer may not have the authorization to use skype…. so wearing good pair of shoes is a plus !
Don't try to do too much
There are a lot of exciting profiles and testing them is a lot of fun, but also a hassle, especially when things don't work out or you have to fix some bugs. You should limit yourself to something that's realistic so you won't have a lot of unfinished profiles at the end. To give you a number, my partner and I did about 50 tests per person in 4.5 days, which amounts to about 11 tests per person per day. That might not seem like a lot, but it really is. I think that trying to do something like 40 tests per person for the week is an achievable goal, but you should be careful not to overburden yourself.