The last TdT meetup was somewhat of a special occasion. For one it was a workshop instead of a presentation, which means that we formed groups to work together then presented and had discussions about our work that we had just done (instead of ONLY speaking from experience gained prior to the discussion). It was also “owned” by four people (instead of the typical one or two) which helped a lot with facilitating the discussion while we were working and as well as after, while discussing challenges and insights. I also had the opportunity to work with people I hadn’t had the chance to, which is always nice :-).
The workshop was structured in a fairly typical way, we arrived at the venue, chatted about random things, I found and solved a rather interesting puzzle game in the kitchen :D, then Dolly, Dorel, Elena and Oana, the content owners, introduced us to the theme of the meeting – learning! – talked about their goals with the workshop – wanting to become teamSTAR this year – go team go! – then described touring and how it fit with the theme.
They also introduced us to the app we were going to test, a chatting platform named slack, similar to an IRC client named – or so I thought before the workshop. They had cue cards printed out with the different tours we could choose from – they went with the FCC CUTS VIDS tours.
Testing tours are not really new, but I’ve rarely seen them being used on projects I’ve been on or talked to other testers about, not to mention being used consistently (this statement is of course limited by my experience :-)). I first heard about touring while doing the BBST Test Design course where one of the techniques we covered was touring, (and where I first read this article by Michael Kelly). While this was my first contact, I only started to understand the power of touring after reading James Whittaker’s take on touring described in his book ‘Exploratory Testing’ (he also has a couple of articles dedicated to this technique here).
I read the book while I was on a project testing medical software and this technique helped me a lot in learning, mapping and better understanding the intricate connections between elements of the product, by touring the application and mapping the central data elements it worked with, then we used this map to vary the data we were working with, after which we moved on to touring the application to figure out what features were important for different types of users, then we created scenarios based on these user types.
While using tours proved invaluable on this project, on my current project I can’t really use the technique consistently (I’m helping in setting up an automation environment and writing automated checks, with a bit of testing sprinkled in when release candidates are promoted).
Given my current situation, attending the workshop was an excellent opportunity to use the technique once more. Participants formed groups of 2 to 4 people and chose a tour type. I had the pleasure to work alongside Tamara and Florin, both fairly new to testing, but eager to start. We chose the Variability tour, because it seemed easy enough: just look for things in the app that you can change and change them. We were somewhat wrong about this.
After choosing the type of tour, we quickly set our laptops up in the corner of the garden 
outside the venue, and designated roles for each of us: Tamara sat at the keyboard, she did most of the navigating and the “trying things out”, Florin did research as well as compared what we found in the desktop version of slack with his Android version, while I took notes and came with suggestions for what elements to vary.
Our first challenge came in the form of trying to figure out how much to vary the data we were working with, while still learning new things about the app. Our reasoning was that since the goal of the Variability tour was to learn the capabilities of the application by changing the different elements it is comprised of (buttons, settings etc.) we should see these different changes take effect not just compile a list of possible variations. So we limited the amount of variation in the hopes to cover more ground.
We started our tour with the different ways Notifications can be set up in the app. Right from the start, there were a lot of ways to customize it:
- Do we want to receive notifications from everyone or just a certain group? How about specific channels within a group?
- Do we only want direct notifications and a select group of highlighted words?
- Or possibly from specific users?
Next there were specific ways the notification popup window could look and feel:
- Should it contain the text being received?
- Should it flash the screen?
- Should it make a sound? If so, which sound?
- Where should it appear on the monitor? How about if there are multiple monitors connected?
We also learned that users can set Do Not Disturb periods (based on Time Zones only reachable from the web version of the app), as well as timers that can act as a temporary DnD when we want to do something uninterrupted. These settings, of course could be further customized on a group/channel level.
After we were done with Notifications, we moved on to varying the look of the app by changing its theme to a different preset, creating a new preset altogether, we even looked at importing themes into slack. We found out that this option is NOT variable in the mobile version of the app. When we started focusing on Advanced options to see if we could find any interesting features, we realized there were only 10 minutes left of the session, and we were at the 3rd point of a 6 point list of settings.
Managing time turned out to be our second challenge.
Needless to say, we tried to wrap up the session by finding elements that were more ephemeral than the settings of the app. We noticed another team had enabled TaskBot, so we varied the different elements of a task created with the bot, created temporary channels made out of our group and tried to send different types of messages. We also ended up talking about the way we varied the messages sent throughout the session so we added that in our notes as well.
After the one hour testing session we gathered inside, where each team talked about their brief experience using tours, and whether they see this as something they could do on their current project:
- There were a few feature “tourists”, who had interesting takes on what to include as a feature and what to exclude (my take on this is that while it’s important to have a common understanding of a feature on a given project I am on, this might change if I move on to another project)
- There was one group that picked scenario tours, who had a hard time coming up with realistic scenarios – or at least they did not talk about the ways they tried to get to these scenarios. (my conclusion was that understanding the features and the users of the app would greatly help in this, so touring the app from these perspectives before searching for scenarios would probably be a better way to learn the app)
- There was a group who did a Configuration tour, which was fairly similar to our Variability tour, since they are related in concept. (my take on the difference between the two is that while in the Variability tour you learn about the application by varying things in the app, the Configuration tour focuses on the permanent changes that you can “inflict” upon the app).
- Lastly, there was a team that did a Testability tour, and they hardly dealt with the application itself, and looked for logs, access to local databases, controls within the app to use to automate different checks with, possible bots to use for creating/modifying/deleting key elements of the app. This tour, while very different and interesting, was the least context dependent of all of the tours (making the experience more easily transferable to other projects)
While we didn’t get to discuss about it during the workshop, a third challenge I see with ongoing tours is this: Do we keep the map we created up to date with new changes being added to the app? In other words, is the artefact we create during a tour lose its value? I know it’s valuable for a while, but is it worth it to keep the map/list updated? As I see it, the act of creating the map or list is vastly more valuable, and it keeps its value while you base other tours on it, I am more skeptical of its usefulness after that, though.
All in all, I came away from this workshop with interesting ideas, and a firmer grasp on the technique than before. If i were to sum up the experience of touring an application and engaging in the discussions after in one sentence, it would be this one: Without resorting to assumptions and our previous experience with the application, it will take a lot of time to do each type of tour, and that some of the tours are more useful to do early (Testability, Feature, User tours) while others are better done later (Scenario tours), but ultimately it’s worth the effort.
