Feature: As an advertiser on a given social media platform, I want to target the types of users that my ad will be shown to.

Seems easy enough. Immediate tests that come to mind are around the advertiser being able to specify the types of users that should be targeted while creating the ad. Sure, this should certainly be tested and won’t be too difficult to validate. However, the challenge comes in when attempting to validate that users who meet such criteria are the only ones being served.

In this talk, Angie will share her story of trying to automate test scenarios that were essentially untestable due to various limitations and restrictions. Warning: this story is not filled with warm fuzzies and rainbows. No, it’s filled with challenge after challenge. Angie will discuss the technical challenges, and how coming up with solutions to these were still not enough. She’ll share the details of her harsh realization that the power to test the untestable was not in her hands alone, and that a “whole team” approach was the only way to truly be successful.

Slides: to be added soon…

Video: A Tale of Testing the Untestable

Sketchnote by Marianne Duijst:

What builds and maintains our culture? Everyday we identify ourselves in groups that categorize us based on practices, rather than shared principles. In the workplace, company values define culture that is deeply rooted in hiring practices, incentive programs, and management. Although a common approach, this is not the way diverse and inclusive culture evolves. There is a loud call and demand for diversity today. So what do we do?

This is not a light topic. Diversity is often met with fear, frustration, defensiveness, and overall exhaustion. So, let’s take a moment to listen. When you do, you will be provided an opportunity to get answers that you may find incredibly difficult to ask.

Through this talk, Ash will take you on a journey. She will help you understand the impact of spaces lacking diversity by sharing accounts of the structural barriers she faced when moving to infiltrate the tech space. She will recall her struggles with assuming “like-mindedness” in order to gain access to the opportunities in tech, as well as highlight what it means to feel included as an “minority”. Most importantly, she will share stories of the residual climb she encounters to be seen as a tech professional in today’s tech space.

“I am not a cultural fit. And likely never will be.” Ash will awaken you to the complexity around culture and opportunities within tech, from the perspective of a Black Woman. And in exchange, you will have the floor to ask the more difficult questions we are all bushing up against. Am I making an impact on either side of this problem? What can I do to contribute? Why should I care about diversity? In this talk you will be challenged to take a hard look at your own participation in these communities and gain understanding of how to recognize issues and support solutions.

Slides: I Am Not A Cultural Fit - One Black Woman’s Account Through The Tech World

Video: I Am Not A Cultural Fit - One Black Woman’s Account Through The Tech World

Sketchnote by Marianne Duijst:

After a 10+ year break from developing I dipped a toe back into the coding world through a series of two-day stops on a “coding tour”. Before I knew it everything was turned up to 11 and I had become a permanent member of a team who Remote Mob Program every day.

How did it feel to return? How does remote mobbing work? Aren’t you autistic - how on earth did you cope with mobbing?

Learn the answers to these questions and more, along with some hints and tips for learning, mobbing, working remotely and tuning for inclusivity.

Slides: Leaping back in to the code via Remote Mobbing

Video: Leaping back in to the code via Remote Mobbing

Article: Experiences from Remote Mob Programming: Q&A with Sal Freudenberg by Ben Linders

Sketchnote by Marianne Duijst:

The complexity in complex distributed systems isn’t in the code, it’s between the services or functions. And a lot of failures are hard to predict and maybe even hard to detect.

When your system is made up of multiple microservices or a bunch of lambdas and some queues, how do you test it? How do you even know whether it’s working the way you think it should?

Quality in these systems isn’t so much about testing up front: if you’re releasing 20 times a day, you can’t pay the cost of running full regression tests every time. You need to have a risk-based approach and focus your testing effort on the things where it really matters. And more importantly, you need to be able to quickly find out when things are going wrong, and quickly fix them.

Your production system is the only place the full complexity comes into play, so you should be doing a lot of your quality work there. Make sure you can find out about problems as early as possible and do as much ‘testing’ here as you can.

I’ll be talking about the importance of observability of your system - building in log aggregation, metrics and tracing so you can tell what’s up. I’ll also be talking about business-focussed monitoring, including synthetic monitoring.

I hope to show you why it’s worth dealing with the additional complexity of microservices over the monolithic approach of before, and give you some ideas about how to make your complex distributed systems easier to build and to run with high quality and stability.

Slides: Quality for ‘cloud natives’ - what changes when your systems are complex and distributed?

Video: Quality for ‘cloud natives’ - what changes when your systems are complex and distributed?

Article: Testing Complex Distributed Systems at FT.com: Sarah Wells Shares Lessons Learned by Ben Linders

Sketchnote by Marianne Duijst:

Are you a strong exploratory tester that up to now hasn’t had the opportunity to learn more about API’s? Or perhaps you are an API test automation whizz but you want to learn some API exploratory techniques?

This class is not only about learning a new skill, it’s about collaboration. By pairing exploratory and automation expertise together, we will have the opportunity to coach and learn from each other. Time permitting we will discuss how this coaching (and learning) felt, what lessons we learned and what we would do differently next time.

The workshop will begin by focusing on exploratory testing at the GUI level and developing a model and then questioning our model. We will then move to the API learning how to use chrome developer tools and postman to send and receive messages using a REST API.

We finish by performing some API exploratory testing to discover the worst bug!

Workshop Requirements: This workshop requires you to bring your laptop, with Postman installed, as well as your favorite mind-mapping tool (for instance, XMind or pen and paper).

Slides: Coaching & Learning about API Exploratory Testing

Testing for web accessibility can feel daunting when you aren’t used to it. But with the growing requirements for sites to meet accessibility guidelines, it has become an unescapable part of a developer’s life. Accessibility should be seen as a challenge, rather than a chore.

This workshop will demonstrate an accessibility testing workflow that can be integrated in coding workflow, and will include a review of automated testing, and hands-on manual testing using a variety of tools, from using the keyboard to using a screen reader application. We will work on real life sites, as well as some code snippets.

Participants will leave the workshop with a solid understanding of common accessibility barriers and methods to test for, and remediate, these barriers.

Workshop Requirements: Details on requirements are available here: https://incl.ca/speaker/practical-hands-accessibility-testing-preparation/

Slides: Practical Hands-On Accessibility Testing - Thurday
Practical Hands-On Accessibility Testing - Friday

Software teams often don’t control all the pieces and parts of their development or delivery environments. Especially in large organizations, multiple teams, each with different focuses and goals, are needed to deploy new and updated features. Seemingly unimportant details have potential to derail a project or a design, before you’ve had time to adjust. How can you and your team become aware of external impacts, and find ways to get value out to customers? In this workshop, Melissa Eaden and Lisa Crispin will shine a light on organizations. They’ll help participants learn to use dependency mapping to identify all the other people, resources, external organizations and more that can impact a software delivery team.

Goals:

• Learn different kinds of dependency mapping and the visual and informational value they provide

• Learn interview techniques to you can use to collect data about your application’s dependencies, and how to turn that into visual maps to help your team manage those dependencies.

• Practice interviewing and creating maps to help identify relationships you need to build with other parts of your organization.

Slides: Using Dependency Mapping To Enhance Testing Techniques

How do we ensure the feedback we are getting from our automated tests is targeted and informative? What information are we throwing away in an attempt to check for specific data?

How we design assertions and the tools we use to determine the value of our automated tests. However, most of the time we neglect our assertions, relying on libraries such as Hamcrest, Chai and Assert. Enter approval testing, a different approach to assertions that can improve tests feedback loops. By increasing the scope of what is being asserted without sacrificing speed, reliability and maintenance, approval testing can help superpower your automated tests feedback.

In this practical workshop, attendees will learn the how and why of approval testing techniques by creating automated tests using approval testing against different application layers.

By the end of this workshop students will be able to:

• Discuss the goals of automated regression testing and feedback loops
• Describe how approval testing works and differs from traditional asserting
• Construct approval tests for different interfaces ranging from API to Visual
• Construct methods to ignore specific data during approval testing

Workshop Requirements: Details on requirements are available here: https://automationintesting.com/setup/settingupapprovaltests.html

Slides: Approval testing - Superpower your automation feedback

“SHIFT LEFT!” “CI!” “FAIL FAST!” “TESTING IS A CRITICAL THINKING EXERCISE THAT HELPS US BETTER UNDERSTAND OUR SYSTEM AND THE IMPLICATIONS…. sigh…”

Let’s be honest, truly transformational ideas are rarely easy to get started, but some ideas are certainly stickier than others. We have seen some excellent changes happen in our industry that have really advanced our ability to build software faster… but where are the sticky changes that really focus on quality and testing? We have some of the worlds great thinkers writing about this. The world demands quality… but it still feels like a dogfight every time.

Inevitably, it becomes a prolonged (and ongoing) discussion point rather than a movement. Is there any way we can take advantage of the power of momentum? Is it all about clever terms and catchy phrases? Or is there more to marketing and branding that we just are not taking into consideration?

Is there a science to branding? Short answer… yes. And we really need to start to pay attention.

In this workshop, we will explore what the world of branding and marketing can teach the testing community. Areas of exploration include:

• The science of measuring market energy and disposition
• Performance as authenticity
• Incremental idea injection and experimentation
• Researching market categorization
• What the world of online dating can teach us (no really… this is super important)

Bring your challenges. Bring the testing transformations you want to see happen at your workplace. Bring your frustrations, your hopes and your concerns. Together, let’s explore what the world of branding might share with us, and tap into some of the worlds most effective methods for connecting great ideas with real energy for change.

How did we find that bug? What does it mean when we say “we had a feeling that was going to happen”? Can we get more concrete at describing how our brains are working when we do exploratory testing?

Yes, we can. In this talk I’ll talk about the wide variety of heuristics we’re using when we’re doing exploratory testing. I’ll bring examples of ones I’ve noticed and talk about how we can all go about identifying our own and others’ heuristics.

Join this session if you want to improve your exploratory testing, get better at describing what you do, or want to find ways of teaching others about it.

Slides: Heuristics and hunches in exploratory testing

Do you code on your own? When you do, do you write unit tests? Do you aim for test-driven development?

Recently, Sal Freudenberg and I have been remote-pairing on an iOS puzzle app. I love to play this game we’re building, which makes me not only software engineer but also end user, product owner, business stakeholder, and everything else. As a user I wanted new features and I wanted them NOW. I didn’t care about the tests - I just wanted a better game!

The reason I asked Sal to join me is that my testing strategy was getting increasingly lax, and the quality of the code was suffering. The increasing lack of tests made the code harder and harder to extend.

I stopped testing for bad reasons, but started again for good ones. The experience has allowed me to demonstrate in practice the impact it can have on your code base when you don’t test, and also when you do. This talk, based on simple practical examples, will explain exactly why unit tests, a TDD approach and pairing can make all the difference to your code.

Slides: How to Stop Testing and Break Your Code Base

Since many organizations are moving their applications to the cloud, investing in some kind of service distribution or even applying a microservice approach, gathering relevant information about the system behaviour is becoming more and more critical. Usually, proper services are available in our production systems to get insight as to what is happening with our production systems. However, this information might be relevant as well at our testing environments. How can we bring the degree of detail available sometimes only at production to our QA environments? This talk will discuss tools and practices that we can leverage to acquire relevant information from the product and its tests, and how this information can be put together to facilitate a joint work between testers and developers not just finding bugs, but also finding the root-cause of those bugs, making everyone’s work more effective.

Slides: Bringing production observability to your testing environments

Article: Observability in Testing with ElasTest

In education, we think about every student as someone with the potential to do amazing things, and as teachers, we adjust our strategies to guide our students to the best outcomes for their lives. Management is no different! We absolutely want our teams to receive the best guidance and support that we can offer. This talk will explore the concepts of “differentiated instruction” as they apply to managing and leading a team, so you can make sure that no team member is left behind.

Slides: to be added soon…

Article: Great Managers Are Like Great Teachers: Q&A with Jessica Ingrassellino by Ben Linders

There is a lot of thinking that test automation will help you release faster and with better quality because you can catch bugs quicker than any manual processes.

It’s almost as if it’s a cure to our problems within software development. But what if we could take a more preventative approach to these ailments and stop the bugs from ever developing in the first place?

For our teams it’s looking at our development practices as a lifestyle. To be reviewed regularly, with plenty of rest and regular check ups to make sure we’re on the right path.

This talk is split into two parts with the first detailing our journey with test automation and the second half about what new things we have introduced to start heading towards a more productive and healthier (development) lifestyle.

Searching for a cure to our problems: From weeks of manual testing and only releasing a handful of times per year. To using frameworks such as BDD to understand what it is we’re trying to build and then automate our acceptance testing. I’ll talk about how we went about applying BDD principles and what we really got out of it.

Taking a preventative approach: I’ll cover some of the custom software metrics we employed to understand how we are building our systems and the behaviours this influenced. Our use of pair-programming to speed up code reviews, addressing our technical debt weekly and our renewed focus on code level testing to build testability into our systems.

Take aways

• Strategies for testing and automation
  • What you should be using testers for
  • How to focus your automation on core journeys
  • How to improve your unit tests
• Approaches to improving your development practices
  • Software metrics
  • Pair programming
  • Unit testing
  • Handling technical debt

Slides: Test automation - Cure Vs Prevention

The best interface is told to be the one that is seamless to interact with, and a new wave of interfaces based on text, voice or gestures is emerging based on this motto.

At the same time, these interfaces make use of advanced algorithms, ranging from pattern and expression matching engines to natural language processing and AI algorithms.

Given these starting conditions, testing conversational interfaces can raise challenges, from both the novelty as well as from the complexity point of view.

My session covers the most important aspects of testing such interfaces, sharing experience on what to do and what to look for, highlighting the importance modelling intents and transitions in the state-machine model

Key learnings:

• What is a conversational interface?
• How can I build my testing approach for such an interface?
• What from my current toolset can I use for this new context?

Slides: Conversational interfaces - Testing the bots!

Microservices are hot! However, with many small services there are also many integrations. As we all know, integrations are a hotbed for bugs. End-2-end tests could reduce that risk but it also introduces a new monolith: the tests itself.

Fortunately there is another solution: Contract Testing. A lean way of describing and validating contracts between services. In this talk I will share my experiences with Contract Testing:

• What is contract testing?
• Why do you want it?
• My lessons learned

Slides: Contract Testing - bye-bye Testing Monolith?

Article: Using Contract Testing for Applications With Microservices by Ben Linders

In this talk I will take you through the challenge of testing a cloud-native application that uses MQTT (a pub/sub protocol), webhooks and REST to interact with IoT devices on top of AWS. It uses services which cannot be run on a developers machine for testing, so I needed to develop a test setup which enables us to continuously test against real services. On top of that I wanted to use a BDD approach to write the tests to make our progress in the project self-documenting and the test immune against effects of big refactorings, which provides an extra set of challenges since these tests are written in a serial manner, which does not fit the async execution of the real solution. I’ll show how all this is done using JavaScript and that this is is not only an interesting challenge but uses tests that behave like real users, and that there is an Ops person in all of us.

Slides: It does not run on my machine - Integration testing a cloud-native application

Test Driven Development has established itself as a cornerstone of sustainable product delivery. However, what we accept as a common practice for writing our applications has yet to be adopted as usual practice when it comes to infrastructure. In this talk I intend on showcasing a way of building application containers through unit-, functional- and integration testing using ServerSpec, Ruby and Docker(-compose) as a foundation for Continuous Delivery. This will include a live coding example, as well as a step by step introduction and ready-to-use sample code. In the end, hopefully, you will feel empowered to build your on containers using TDD, while playing port authority whenever you ship something to production.

Slides: to be added soon…

Mobile test automation was born at the beginning of mobile apps development and became popular for the last 5 years. It existed for a while already on big projects in big companies. However, the majority of mobile developers have been raised with a manual testing mindset and it’s a huge challenge for us, test automation engineers, to change that attitude. Here I’d like to share my 7 year’s journey in mobile test automation.

• Why mobile is different than other platforms (web, backend)
• What are the tools being used and how are they different
• Where mobile automation can go wrong
• And finally, how to fix it

Slides: Ways to fail at mobile test automation and how to avoid it

Article: How to Avoid Failing at Mobile Test Automation by Ben Linders

Domain invariants are all around you. In every business rule your domain expert ever tried to give you. You should use them to guide your design. You should also be testing them! Not only an example representing them: but testing the invariant itself.

You can do them with property-based tests (PBTs). If you think you cannot do PBTs on legacy code bases (or outside FP) this talk should show you otherwise.

Let’s also use property-based tests to reduce test-debt. Create smaller, fewer tests that: test more, are more readable & document the problem. Challenge your understanding of the domain and communication with domain experts.

Have you heard about TestSphere? It’s a deck of cards designed to get you thinking and talking about testing. I was inspired by them and wanted to use them more in my everyday work. So I decided to start a “30-day TestSphere challenge”: for each of my testing tasks, I randomly pick up a card and try to apply the idea from it within my testing activities. I’d like to be honest and in the talk I will present which ideas worked and which didn’t. I hope that my example will inspire the others to think wider, be more creative and effective in testing.

Slides: My 30 day TestSphere challenge - how was it

These will be small 8 person discussions, facilitated by a speaker. Their purpose is to get you talking about the topics that you care about with your fellow conference attendants. For those of you familiar with kanban or scrum board, the structure of this activity will feel familiar. For the rest of you, you can learn about it here.

At the end of the conference we do an Open Space. This is your chance to get exactly what you need from European Testing Conference. Open Space sessions can be on anything and some of the best ones have titles like: “I am having trouble with ___ Please Help!”

More details.

This is a session to get everyone introduced to at least 10 new people. You will draw a personal mind-map (with nodes: personal, work, accomplishments). Every 5 minutes you will exchange mind-maps with a new person, and each of you will ask the others about the items you are interested in. Afterwards, the conference will be an even friendlier place.