Author Archives: offbeattesting

As a salaried employee I believe that I have an obligation to my employer for a certain level of productivity.  The way I look at it, I’m not getting paid for how many hours I clock in a week, I’m getting paid for how much value I can give the company.  This means I have a responsibility to be productive with my time.  There are two sides to this.  One is the idea of how much stuff you can get done in a given amount of time.  The other is figuring out if you are doing the right thing.  It is quite possible that you are very efficient at getting things done, but if you are not doing the kinds of things that add value to the company you really aren’t being productive.

On the other hand, you could be well focused on the kinds of activities that add value to the company, but working on them in a way that keeps you from efficiently getting them done.  We need both things, but in this post I want to talk about the idea of getting stuff done.  Specifically I want to talk about some of the productivity approaches and tools I have used and the kinds of things that didn’t work for me as well as some of the approaches I have found to be effective.

What didn’t work

I’ll start with some of the things I’ve tried through the years that haven’t seemed to work for me.  It is pretty easy to find ‘productivity hacks’ on the internet, but there seems to be a lot less talk about the ways some of these things don’t work for some people.  We are all different and so something that works for you might not work for me (and of course, something that didn’t work for me might just work for you).

One thing that I have tried is the pomodoro technique.  I used it for several months and there are some things I liked about it, but at the end of the day is just didn’t work for me because of the amount of time I spend on collaboration.  I think the technique would work well if you are primarily working in focused time on your own, but I am frequently stopping to answer questions and help other testers or developers, or I am asking questions of my own.  One of my goals is to see testing and development work be more closely aligned and this means an emphasis on collaboration.  I found the pomodoro technique shifted the emphasis too much to my individual work and away from a team based approach to solving the problems at hand. I have, however, been able to apply some of the thinking of this to help with my productivity.  I now schedule a deep work session each day where I have some uninterrupted time to focus on work that requires that type of thinking.  This allows me to balance out responsiveness and collaboration with thoughtfulness, introspection and deep thinking.

Another thing that has not worked out for me is using trello boards or kanban to structure my work.  I like the theory of it and I continue to try and implement the idea of limiting work in progress, but I found that it create too much ‘paperwork’ for me.  I have a strong dislike of paperwork of any sort and creating and maintaining a backlog and moving work through various stages, just felt like too much overhead for me.

What did work

There have been some things that I have found did work well for me.  The system I have found to be most helpful is summarized here. The basic gist of it is that information, communication, scheduling and task management are all different things and ought to be thought of and managed separately from each other.  While I don’t use the same set of tools, that system of thought – breaking those things out into separate tools and managing them each independently of each other – has led to a much more productive approach to life.  Fundamentally this approach to getting things done in a digital culture has transformed the way that I work.

When talking about things that worked well for me, I have to gush for a minute about todoist.  I’ve tried a number of different task management systems over the years and this one blows anything else I have tried out of the water.  It is simple and powerful, probably because it does only one thing – manage tasks. It allows you to manage your tasks in an easy and intuitive way, and best of all for me, to instantly add something as soon as I think about it.  This keeps me from forgetting things and also allows me to quickly add a task without it disrupting  the current work I am doing. And no, I was not paid to say that – this is a tool I have come to love for their disciplined approach to solving just one  problem.

Another thing that worked very well for me is the fact that I have tried many different approaches and tools.  By playing around with different ways of doing things I have been able to learn things from various systems.  I have been able to learn what things work for me, what things don’t work for me, and what things need to be tweaked and changed to work for me.  I have also been able to recognize that some things work well in some contexts and not in others.  Since there is no one size fits all approach to getting things done in knowledge work, experimentation to find out what works for me has been key.

So what kind of approaches and systems do you use in your attempts to get things done?  How do you approach that digital mountain in front of you?

Debunking Handbook -This is missing one of the key factors in changing people’s minds – trust and relationship –  but it is still a helpful way to think about more persuasive fact presentation.  Good ideas here for writing defect descriptions for example.

Disorganized Thinking and Deep Work Challenge – two different views on how to be productive.  Should you fight for focus or not? I think both of these can be applied at different times

Simple Models – Keep you models of the product simple and build a lot of them.

Testing Kraftwerk – Some other thoughts on how to build models

Who doesn’t like to complain about meetings?  If your job involves creative knowledge work you probably find that meetings interrupt your flow and keep you from getting the things done that you need to.  I’ve had a long and active resistance to useless meeting and I’ve tried to vote with my feet as much as I can, but I was recently thinking about which kinds of meeting I usually find to be wasteful and I think I have come up with a heuristic for identifying wasteful meetings.

If a meeting is run on a regular schedule and the time slot is longer than 1/2 hour, the meeting is most likely wasteful

In my experience, it is often ok to have a longer meeting if it is called on an as needed basis, and it is ok to have a regularly scheduled meeting if it kept short, but as soon as you have a meeting that is both long and regularly scheduled it will end up including a lot of waste.  Team work is vitally important in knowledge work and you can’t have teamwork without meeting together.  So it is not meetings themselves that are the issue.  The main thing is understand what effective team work looks like.  When the team is too big it rapidly changes from team work  to team discussion (or worse, arguments).  If the time slot for a meeting is longer than 1/2 an hour it usually indicates that there are a lot of people in the meeting.  There are times when larger groups need to get together and have discussions, but in the day to day work of the knowledge worker those times ought not to be common.  This is why we should not have regularly scheduled meetings with big attendee list – they end up being wasteful.

What about you?  have you experienced this with meetings?  What kinds of meetings do you dread and try to get out of going to?

You may have heard about the dangers of having a hero culture in software development.  The reason this is seen as a problem is that we don’t want to have one person on the team with a great deal of specialized knowledge.  We want to make sure that the knowledge is spread around the team and that we have people who can work in a variety of different roles and places.  I agree with this, but I think that there is still a place for heroes.

What we need is a certain kind of hero.  We don’t need the heroes who take on work so that everyone will come to depend on them.  We certainly don’t need heroes that look down on the lesser mortals on the team and will do it for you but not teach you how to do it.  What we need are old fashioned heroes.  The kind you read about or hear about from war stories.  The kind of person who jumps on a grenade for his buddies.  We need heroes that care about the team and will sacrifice themselves for the sake of the team.  We need heroes who work on reducing negative influences on the team.  We need heroes who know how to build and forge strong relationships.  We need heroes who are willing to do the dirty work and who are willing to share everything they know so that the team can succeed.

In short we need to talk about heroes as those who sacrifice for others.  In many ways superhero movies have taught us that a hero is someone who can save the world on his own, but we need to celebrate a more simple style of heroics on our teams.  The real hero isn’t the one who checks in a fix to an important bug at 2 in the morning.  The real hero is the one who stopped to to chat with the developer at 2 in the afternoon and triggered a thought process in her head that lead to the bug never showing up in the build.  The real hero is not the one who stayed late all week finishing up the new feature so that it could get shipped in time.  The real hero is the one who spent a half hour pulling the team together to explain what was going on so that the team could work together on coming up with a meaningful solution to the problem they were facing.  The heroes we want are the ways that make the team better and that you almost don’t even notice.

Do you know any heroes like this?  Do you celebrate them?  Are you one? Viva La Hero!

Exploratory Testing Self-Management – Good exploratory testing is hard. Managing it can sometimes be even harder.  Some really good tips here.

Test Tradeoffs – Think about your tests in terms of what you want them to do, instead of just what kind of test they are.  This one has really got me thinking.

Mentally Owning a book – Sounds like a lot of work (maybe more than it is worth), but do you mentally own your books?

Password Managers (video) – What password manage do you use?

The job of a tester is not just to find issues, but it is also to present the case for why those issues matter.  This is much the same thing as saying that we want to find relevant issues and see the quality of the product improve.  There is another side of the equation though.  As with most things in life, trade-offs have to be made and the trade-off here is the balance between how important and relevant an issue is and how much work it will take to fix it.  Some things may be important to fix, but may end up being too expensive to fix and so don’t get addressed.  This can be very frustrating for a tester who sees the importance of the issues found but perhaps has less exposure to the cost of fixing it.

However, something that struck me recently is that there are certain kinds of feedback that are more important to give early in the process.  Some types of issues become more and more expensive to fix as you get later in the release cycle. In my experience workflow and usability problems typically fall in this category.  Early in the development cycle of a feature many things are changing and so it is actually much easier to change workflows or usability issues.  As the feature gets more mature, other things start to depend on it being a certain way and APIs are defined and there is upgrade code in place for older versions, etc.  Making workflow changes at this point in the process is often much more painful and so the cost side of the equation goes up.  As a result, we often have these kinds of issues dismissed as not worth fixing or as a system limitation.  However, I have noticed that if we find these kinds of issues early in the cycle they are much more likely to be addressed.  The benefit side of the equation stays the same, but the cost side has gone down and so the issue gets corrected.

Based on this experience I have been wondering if I need to shift my testing approach a bit and really focus on finding workflow and usability issues early on.  Typically my approach has been to bug hunt at the early stages and try to give the developers feedback about bugs early so they can fix them early. I still think this is a good approach, but what I think I need to do is shift my search to be for certain types of bugs.  I need to look for the usability and confusing workflow issues early and leave the testing for coding and logic errors and crashes and other things that are clearly defects until later in the cycle.  These other items are things that usually have less cost increases as development progresses and so it is ok to find them later in the process.  We need to ask questions early, but we need to make sure we are asking the right kinds of questions as well.

Read more books  – Most advice on how to read more books is just silly, but here is some actually helpful advice on how to  become more of a book reader.  I love reading blogs, but I think that to really advance in an area you have to get into the books.

Discomfort as a tool for change – A lot of interesting thoughts and insights in this one.  The biggest takeaway for me was the whole mindset towards testing.  What are we really trying to achieve and is the testing we are doing working towards that?

Including this just for this quote “At first it was done so that nothing worked. Then it was done so that simple and straightforward things worked. Then it was done so that most things worked” – Sounds like the definitions of done I hear a lot 🙂

Pouring Molten salt into water (Video) – Just for fun.  Always fun to watch an explosion and it’s even better if you can think that you are learning something along the way.

Today I was trying to add something onto my cell phone plan, and I found a bug on the website.  To be fair, I think everything was working ‘as designed’ but it was still an annoying bug for me.  I have had similar experiences the last few times I have tried to do things on their website and this bug even had me toying with the idea of finding another phone provider.

So what was the bug?  Well, I was on the ‘plan summary’ page and there are three things you can do. (1) change your plan – I’m not interested in this so we’ll leave it aside. (2) a Manage link shows up beside the monthly add-ons and (3) an Add now link shows up beside something called Roaming add-ons.   What I want to do is add a long distance add-on to my plan, so I decide to click on the Add now link.  It takes me to a page that has different available add-ons, including the one I am interested in.  I click on the link to ‘add to my plan’ and it takes me back to the plan summary page I started from.  What the @#$%?!  After a few paroxysms and trying on different browsers etc.  I realize that I need to use the link to Manage my monthly add-on instead, because I am trying to add a monthly add-on and not a temporary roaming one.

Now my guess is that everything is working as expected.  When I used the correct link, I had no problem getting the add-on I wanted, but there are some serious workflow issues here.

1. If I click on the Add roaming add-ons link and I can’t add monthly add-ons from the resulting page, they should not show up on that page.
2. The guidance on the initial page should be improved to help me pick the right link in the first place
3. And most importantly of all, there shouldn’t be a workflow that takes me in a circle.  If I started on a page, I should not be taken back to that same page with nothing at all changed.  Either take me directly to another page that will allow me to do what I want, or, at a minimum, give me a message letting me know why I ended up back here and suggesting some possible alternatives for me to try.

So, before this turns into a rant, let’s stop and think about some lessons we can learn.  The first lesson, of course, is that workflows matter.  Everything can be technically working, but still be very frustrating and difficult for the users.  Another lesson is a new hueristic  that I found: don’t go in circles. Are there any circular workflows in your product that might frustrate people?  I will be reviewing my product for this!

A third lesson, which I hope every tester knows, is that people will do things you don’t think they will.  If you provide two paths and you expect that users who want x will use path (a) and that users who want y will use path (b), don’t forget to think about the feelings and perspectives of users who want y but go down path (a).  Just because it ‘doesn’t make sense’ for a user to go down path (a) to achieve y  doesn’t mean your users won’t do that. Anticipate that possibility and help them out that situation and you’ll have a much better quality product.

Looking this post over it is almost a bug report.  Maybe I should just apply for a job at this company 😉

Not Right now –  I would add if someone frequently interrupts you, maybe you need to schedule some time with them so that you can pass on the information they need to know

Detroit’s Patterns of Growth – fascinating look at the development of city streets over time.  Do you see any parallels here to the development of your software framework?

Everything – (Testing tool recommendation) – Best tool I have found for easily searching for and finding files on my computer.

Because we all need to look up and contemplate the world sometimes

I am always amazed by how many bugs can be found with the simplest of setups. For our (physics simulation) software, I would estimate that half of the bugs I find are on a simple cube without doing anything fancy.  However, in trying to think of testing as more than just finding bugs and also as something that can help drive quality in a holistic way, the challenge of finding good (interesting) input data becomes more important. For this kind of testing the important thing is variety and variation (as well as getting as close to customer data as possible).  So how do I find this kind of input data?

There are a few approaches I take to this.  In the first place I can always just create it, and over the last 9 years of testing I have of course created many different kinds of input data that I can use.

This naturally leads to the second approach which is finding data.  One of the challenges of having a lot of different files and sets of input is being able to find the one(s) you need in your current situation.  This is a search problem and I have tried a lot of different tools and strategies over the years to address this.  I have written custom searching tools that dig into the various data sets I have and provide searchable metadata.  I have also tried indexing my machine with google search and other third party tools, but over the years I’ve found the simplest solutions are often the best.  We now have a simple database that searches for files of certain types, opens them and saves an image of them.  This image along with the file location and a very limited set of metadata about the file is displayed on a internal web page.  There is some very limited text searching available, but the main power of this database is to give a rapid visual idea of the what you might expect a particular data set to be good for.

It is quick, dirty and simple, but therein lies it’s power.  It’s leveraging something the human brain is great at (rapid processing of images) to help us testers accomplish something.  Sometimes I spend too much time on trying to come up with elegant complete solutions when a quick and simple solution will give most of value in a fraction of the time.  I was struck by this again yesterday when trying to find some customer input data.  The first thing I wanted to do was to construct some complex queries to find stuff in the support system, but instead I decided to whip together a quick and dirty script that would search through the shared drive for files with a particular set of keywords in their names.  Will this miss a lot of potential matches?  Of course!  People often don’t name things as I would expect.  But did I quickly find a bunch of data that I needed with a minimum investment of effort?  Yup.

So I guess the moral of the story is that although there may be times when the complex solution is the answer, you might just be able to get by with a simple approach or tool instead.  Keep it simple.  Be efficient.