I have been a tester for more than seven years and I grow from SDET to SDET II and from SDET II to senior tester recently. During my career as a professional tester, I was wondering whether I should switch to developer, whether I can find another testing job in other company, whether our SDETs have better career, and whether we (Microsoft) have too many testers. In this series of blogs, I will share my thought to you by discussing how you can become senior tester and have a better career in your professional journal. The article is written for our testers and test managers. I hope my paper can help you think deeply about testing and tester's career, and best wish you have a better career. This article has two parts, in the first part; I describe the four career path of testers and in the second part I give my advice for our testers. You can download work version of this doc at here.
Note: This blog post is the summary of my previous posts, and the opinion in this blog is only my personal opinion
Part One - The Four Paths of Becoming to a Senior Tester 2
Becoming a Professional QA.. 2
Becoming a Test Architect 5
Becoming a Domain Expert 5
Becoming a Tools Developer 8
Switch Role or Stay. 9
Part II Advises. 13
Be Passionate and Be Motivated. 13
Be Open and Be Broad. 14
Making Big Impact 15
Coding, Coding, Coding. 16
Take Time to Think. 17
Understand the Product 18
Doing Things Differently. 19
Suggestions for Test Manager 19
In the article, I believe that there are four potential career paths for our SDETs. They are 1) becoming profession QA which knows how to do web testing, performance testing, load and stress testing with different kind of testing tools; 2) becoming the domain expert who can use the product you are testing as the end customer. 3) becoming testing architect who can lead the testing and quality assurance for the whole team and the whole company; 4) becoming tools and framework developer who can develop world class testing tools; I will also discuss alternative career paths as engineer, such as switch to Developer or PM, switch the areas you are working on.
In this section, I like to discuss the first career for senior SDET, which is becoming a professional tester. In many companies, we call testers as QA (Quality Assurance), QA exists long before Microsoft have the SDET role. You may wonder what the difference between QA and SDET is. Are our SDET quality assurances?
Let me use the definition of QA at here to start our discussion:
QA stands for Quality Assurance and is the framework that ensures development and manufacture of products such as pharmaceuticals, agrochemicals and medical devices are performed in compliance with regulatory requirements.
It's a career that requires the individual to grow, implement and continually improve quality systems and, no, it's not just another job. In fact, this is a job like no other.
A career as a QA professional means you get a real opportunity to influence working practices and improve standards of quality. It's a position that offers a multitude of options for personal and professional development with a variety of roles available through different projects, processes and locations. It is a genuinely responsible position that requires individuals of real caliber.
As you can see from the above definition, QA is a professional, such as dentist, teacher and it need its own skill. During my whole career as tester, I follow many QA professional's blogs. They are James Bach, James Whittaker, Elisabeth Hendrickson, Cem Kaner, Harry Robinson, and many Test Architects inside Microsoft. They teach me what is software testing, why we need testing, and how we do testing. So what is the common between them? They are both the best QA in the world. They all have very deep testing knowledge, such as Model Based Testing, Exploratory Testing, Testing in Production, Scenario Based Testing. They invented and socialized these ideas, and share to our testers. It was amazing to see how many great testing techniques we have now and how the techniques changes from time to time.
As you can see, coding skill is not a significant factor when it comes to the QA professional, and testing skill does. On the other hand, SDET is one kind of software engineer which specialized to automating test case. In other words, SDEs (Software Development Engineer) write code to make production, and SDETs write code to automated tests. Coding skill is one of the most important skill our SDET should have (I would expect that you will not be hired as SDET in Microsoft if your code skill is not strong enough).
However, being an SDET does not prevent you becoming a QA professional, but you have lot of chances to be succeeded. During our daily work, we have a lot of opportunities to learn new testing methodologies and use in our project and master it. Having a deep understand testing methodologies, and being able to use them in your testing strategy is very important for testing a project successfully.
So how can we qualify as a professional QA? You must
- Know and use different testing methodologies, such as Model based Testing, Exploratory Testing, UI Testing,
- Have deep testing knowledge on some areas: such as performance testing, web, mobile application testing, security testing (or a security expert),
- Be familiar with some testing tools, such as NUnit, JUnit, MsTest, Selenium or commercial QA tools, such as HP Load Running, or VSTS Load testing,
Let me explain that why we need the second and third skills to becoming a professional QA. When a company wants to hire a QA, they usually have a clear expectation of what kind of QA they want to hire. Because the testing methodology for different area/field might be significantly different, so QA tends to be specialized, some QA know deeply about security testing, some QA have experience on web design. Company wants people have experiences in the certain area and be able to address their immediate need. For example, when working on a web site, we might want to hire people who know performance testing, and web infrastructure to help to testing the scalability of the web site. If the people knew popular web testing tools, that will be a big plus. For example, if you know Selenium, one of the most popular Web UI testing tools, it will give you a big advantage on the job market. Working on big companies, like Microsoft, gives you both advantage and disadvantage. In term of the advantage, you will have chance to do different kind of testing, and grow your skill. On the other hand, you might use in-house testing tools (not public available tools). You also might only work on testing a small part of the system (i.e., you are too deep on functional testing on the small component), but less chance to testing the whole system. If you have very deep knowledge on some area, such as security, or performance, you will have better career in the future.
Another important skill to becoming a QA professional is not about having testing skill, but other skills. Imaging that you want to pursue a career in a smaller company or a Startup, having only testing skill might not ensure you have a job. But if you can do other works, such as automatized build, setup web server, create deployment script. You will have better chance to be hired (because you might not do testing all the time, so if you can do many things at the same time, you are a great candidate).
Why we should be professional QA, why not just being SDET forever? Being a good SDET requires both coding skill and testing skill. However, we need to choose an area to be focused on, either coding or testing in the future. It really depends on whether you are passion about software testing, and whether you want pursues it as a career. If you enjoy testing itself, becoming a professional QA is a good career path. Also, having both coding skill and testing skill is one of the strong points when you apply QA position in other companies. On the other hand, I do see many SDETs are not happy with today's job, and I think the root reason is that they have the engineering root, and don’t like testing. If you are really like coding, switching to dev or do more test automation, test library development might be your focus. In the past, we used to have a separated two separated role: SDET, STE (Software Test Engineer), one focus on automation, another focus on testing, and many Companies, such as Amazon, Google have similar roles as well. I believe this might help us in the long term as well.
One thing we might be noticed is from the above definition, we know Quality Assurance is a career for many industries, not only for Computer Software field. For example, in car industry, QA is in charge of examining whether the car we made meets the quality bar. Unfortunately, QA in software industry is still not mature enough. Today, very little colleges have Software Testing as a major or have a software QA degree. Many of us choose SDET as our career because we are hired as SDET, but not we like testing. We saw lot of blog posts and articles about tester (SDET)’s career inside and outside of the company, and we will continuous to see this happens in the near future. It is just because testing as a professional is not mature enough, and I can predict that there will be lot of changes soon (such as Combined engineering in Bing team, and Testing in Production shift tester’s role into more service monitoring/operation area).
In this section, I like to discuss how we can grow to become a test architect. In here, Test architect is not a title, but a role. For example, you might be a Test Architect title if you contribute a lot to testing tool/framework; you might not a test architect. Similarly, some testers play test architect role in their org, although they don’t have a test architect title.
What means test architect role? In here, I list a couple of important areas which a test architect should always do:
- Defining overall testing strategy for a feature, such as how to testing a browser, how to testing a cloud database, etc.
- Being an expert on one of the testing area, such as security testing, performance testing, cloud testing
- Adopting or inventing new testing methodologies to the team, such as explorative testing, crowd sourcing testing.
- Mentoring and grow the team in term of engineering excellent.
- Thinking about the future of testing and tester's career and be prepared for the future.
- involving various actives related to testing, such as testing talk, conference, and blogging,
How can we become a Test Architect? First of all, it is not a trivial task, you need to a professional tester first. As professional testers, you will enhance the above skills during your daily work, and grow towards to being a Test Architect. In additional, many Architects have lots of experiences in different company, organization by doing different projects. By being involved into different kind of project, you can always learn new ideas. In the end, you have the ability of adopting change, and contribute to any projects quickly. This skill is crucial for being Test Architect.
Some test architects today provide consult services to other company. They usually have broad set of knowledge, from Agile Development, Project Management, to Communication Skill and Risk management and help to survive many nearly failed projects. They are the best test professional, and gain respect from the fields.
Today, I like to write one of the most important career paths for our tester, which is becoming a domain expert. We need to realize that many of us will not grow as a tools developer or a test architect in the end. They will become a professional QA, a domain expert or just switch to a new role. Personally, I prefer you consider growing yourself in the area of Domain Expert.
What is a Domain Expert?
Let me answer the question of what is a domain expert. Suppose you are working on testing particular software for five years, are you qualified to be a domain expert? The answer is not really and it really depends on the meaning of domain expert.
For example, I have been working on testing SQL Server for six years. I am quite familiar with the data types, the collations, and can write basic SQL queries. The term "Domain Expert" in this context is that you should be able to design database apps or managing databases. Why I define domain expert as this, because it is the base requirement for a database person to find out a job in other company, which is as a database developer or a DBA. Am I qualifying a domain expert? I don't think so because I only know a small part of SQL Server. I don't have experiences on design a database schema, building an application using database or managing a lot of SQL server instances. It will be hard for me to find a database developer job or a DBA job.
As you can see from the above example, the term "Domain Expert" is really depending on the context. If you are working on Windows team, the "Domain Expert" will be the person who knows installing/configuring/managing Windows, or the person who can write Windows based software. If you are working on Visual Studio team, the "Domain Expert" will be the person who know how to programming using Visual Studio and .Net. If you are working on Windows Azure and SQL Azure, you should know how to build a scalable app using all techniques available in Windows Azure. In this sense, the domain expert need you have a broad skill rather than very deep on small area. It also focuses on end user how to use the software or the service we are testing.
Why we should become a Domain Expert?
At one day, you may consider leaving the current position; you may either choose a different group or a different company. One question you may ask for yourself is that what kind of skill I have learned from my past experience as a SDET, and what kind of position I am qualified. Unfortunate, today many of our SDETs have very deep knowledge on their components, and they lack of broad view of the product that is testing for. One of the reason is that our tester today focus too much on functional tests, which I believe will owned by dev in the end, and less focus on user scenario, or on how our end user use our product. That is the main reason that even I tested SQL Server for 6 years, I am still not qualified as a database developer or a DBA.
You may wonder why we should consider becoming a domain expert or in another way, why not just stay in the testing role forever. The reason is that it will open a very board door to your future and enable you having a better career. The need for domain expert will be much higher than professional QA, and the compensation will also be higher, especially when you become a solution provider.
This is especially true for SDETs in Microsoft. Our company has so many great products, and so many customers. There are high demands on domain expert or professional who knows Microsoft Stacks and know how to build end to end solutions. The more you learn about Microsoft stacks, the better of your career.
Suggestions for Tester
Now, I like to provide some suggestions for our testers. First, ask yourself who kind of person you want to be after three years, want to become a domain expert or want to pursue as a testing profession. This is the question I suggest you think and make decision as earlier as possible.
Second, if you want to become a domain expert, you need to have a plan to grow as a domain expert. Here is some steps help you:
1) Decide which area you want to be focused. We are so lucky being at Microsoft that we have so many great product and so many areas we are focus on. In recent years, IT techniques changes so dramatics, we should carefully choose the area which follow the trend of IT. In here, I like a couple of areas you might be interesting to know:
- NoSQL and BigData is the hot area in database management field. There will be a huge need for people who know NoSQL, such as Hadoop, MongoDB etc.
- Windows Azure is Microsoft's Cloud Computing Platform. Fully understand ingthe system, and having knowledge about building scalable system is a big plus for your career.
- Windows Phone and Windows8 is our next version of operating system. Be able to build apps for these platform enable you find a developer job easily.
- Enterprise customer wants to integrate social network, office, and mobile and Bing search together to provide better productivity. Be able to knowing Office 365, and other offers of Microsoft can enable you become a solution provide.
2) Develop your skill during your work. Once you have an idea of what kind of area you want to be familiar with. You need to develop the skills. If the area you are interested is not the area you are currently working on, consider to switch group. Also, doing side project and get involved in Garage projects by doing grass root innovation is always a good way to build up your skill set. There are plenty of great resources you can take advantage as a Microsoft Employee, and I do suggest that you explore them and build up your knowledge. I do suggest that you set a goal, and continuous improve your skill set. It is your career, and you should seriously spend your time on it. Please view my other blogs for some suggestions that you can improve yourself.
Suggestions for Lead and Manager
Dear lead and manager, I hope you can realize that not all of your employee can be a professional tester in the end. We need help our people to grow their domain knowledge, and give them a better career. One day, when your employee decides to change role or leave the company, they will thank you for providing chances to help them build their knowledge, and thanks for Microsoft provide the platform for them to grow.
Sometime, building a healthy and happy team is more important than finishing the tasks. The great asset of Microsoft is the great people we have. As leader and manager, we should commit to make our employee happy and have a better career. Encouraging people to learn new stuffs, allowing employee to spend their own time on some area is always a good way to grow your employee. You will also realize that by doing so, your employee will also bring some new stuff back to their daily work as well. Having domain knowledge and knowing how customer using the production is always good for testing, and it will be the trend of software testing.
Today, many of our testers wrote testing library and testing frameworks to assist testing automation and lab run automation. We have plenty of testing framework, lab run system cross the company. Writing testing tools is an important skill that it can help our tester grow their code skills. Today, some of testers only develop test framework and test library. In general, you are just as a dev, and you write a couple of codes. The big different between test tool developer and tester is that code skill is the most important skill for developer, and testing skill is for tester.
One challenge for our tool developer is that you should work more with closely to your customer which are the people use your libraries, and make sure you can increase productivity. Please keep in mind, writing tools is not your goal, make other people agile is your goal.
One suggestion I can give to testers who want to a tool developer is that you may look broadly, writing a testing tool is the same as writing other software, so you might have potential big career grow in many team if you have good code skills, so don’t limit yourself for finding SDET jobs or writing only testing tools.
In another view, test tool developer is the same as builder, complier developer, UI developer or database developer; they are just developer who has professional expertise in different areas. That is the reason I put the title of this section as “becoming tool developer” instead of ‘becoming test tool developer.
It brought up another interesting point is that our SDET role is actually the combination of Professional tester and Test tool developer. We want people to do more test automation (tester’s role) through coding (dev’s role). But in some case, we found that we are neither good on both areas. It might potential limit our SDET’s long term career grows.
I was planning to write some suggestion for whether you should stay in your current role or switch to another role in different org or different company. Before I wrote my thought, I think I can give you some references on this topic.
The first article is a 10 year old article published on Interface, the title is "Career?! What career?". The article first states that "Your career development is your responsibility." and "You manage your own career." And then talking about how you, your manager and Microsoft can work together to help you grow. The article provides a series of questions for you to answer. Depend on your answer, it provides very good suggestion whether it is time for change. One of the best part I like is that it has a lot of Probing, open-ended questions. For example,
- What have you enjoyed most? What was it about those times that made them so good?
- Were there times that you especially didn't enjoy your work? Why?
- What accomplishment from the past year are you most proud of?
- Have your ambitions or long-range goals changed since you started your career? When? Why? How would you now describe your long-range goals?
- What are your values (the main criteria by which you judge how you are doing)? Does your current job and your organization within Microsoft match your values?
- How is your manager doing as your coach? Who else would make a good coach for your interests?
- What are you really good at? What do you want most from your career?
- As you look ahead in your career, are there things that you would especially like to avoid? What about these things make you want to avoid them?
- What do you think will happen in the next 10 years of your career?
- What skills or experiences do you need so that you are ready for the job after your next job? For your 10-year plan?
- What can your manager (or Microsoft) do to help you achieve your goals? What do you need from them so that you are successful?
- As you look ahead in your career, what are the things you are especially looking forward to? Why?
- What do you think your next job will be? After that job, what do you think your next will be?
By looking back..., you are asked to thinking about the past working experience. By looking forward, you are asked about what you want to be. Answering these question really help you to think about your career.
Then, the second article is on the same issue which is about "Is it time for a change?". The article listed eight career path options: Enrichment, Lateral, Vertical, Cross-functional, Realignment, Exploratory, Perform, and Other Pursuits. The article discussed how you decide when to make change, When is it not time for a change and how to do your homework and make a good move. It also has a lot of examples from others. For example
If you are unhappy in your job either because you don't like the type of work you do, or because the people you work with have values or a group culture that doesn't match your own, making a change may help, but do your homework first! "Switching jobs isn't a refuge," says Lou Nell Gerard, who transitioned from an administrative position to PM. "Make a move for the positive because you really want to do something, not to get away from something you don't like."
When you have a clear goal and have determined whether or not you can devote the extra effort that may be needed to a new challenge, you will be prepared to see opportunities that match those goals and act on them. For example,
Talk with your manager. Depending on the situation, this could be an invaluable step or one you'd rather not do. Your manager can be your best advocate and support for your change. Although some managers may feel threatened if you begin mentioning things you want to do that are different from what your job is now, a good manager will realize that your growth is a good thing for Microsoft and will try to work with you on your goals.
Sometimes you and your manager simply may not be a good fit. It happens. If you can't talk with your manager, you may want to choose a mentor to help you with your decision-making.
Set up informational interviews. These are informal talks with others about their jobs, such as what they do, how they got there, and what skills are important. You do not need your manager's approval to conduct informational interviews as you do with formal job interviewing. Interviewing in this way actually benefits Microsoft as a whole: you learn more about the career direction that works for you as well as the opportunities that are available at Microsoft.
If growth is your goal… Consider looking for a job on an established team with leaders who have a proven track record. You can learn a lot from working within a well-run organization, including when and how to innovate, when and how to deliver, and good team processes.
If advancement is your goal… Consider teams that are new but of strategic value to a division. These teams start small and grow fast. They can provide you with opportunities to advance quickly and gain visibility. Much like the risk involved with investing in a startup, new teams have greater opportunity for advancement but also have higher risks—many of them never ship anything, and they may require long working hours with less staff.
Look outside Microsoft. Get some perspective from people outside our company. "True career growth is larger than Microsoft," says Scott Berkun, design and usability training manager for MSTE. "The differences and comparisons you can draw may help you to make more informed decisions about what you see around here; in some cases we are more stratified and hierarchical than other tech companies, and in other cases we are more open and flexible." Without outside perspective, you won't see your opportunities within Microsoft as clearly.
When you change to a new position, you want to make a move that is going to further or round out your career. Barbara Wilson, lead training manager for MSTE, suggests three litmus tests for finding out if you are making a good move:
Last job test. This works if you have more than one option (staying where you are may be one of them). Assume that this will be the last job that you will have at Microsoft—which of the jobs that you are considering will support what you want to be doing several years from now?
For example, if you see yourself moving into a teaching role and you have a choice between a job that is really technical and one that involves training others, the latter position might be a better choice.
Can I get excited? test Ask yourself four questions: Can I get passionate about the product or service? Can I care about the customer? Am I interested in the problems this position or team is solving? Does this team's culture or management philosophy fit my style?
Is this a good match? test Ask yourself what the position is going to offer you, and then ask yourself what you are bringing to the team. If the two answers seem complimentary, it is probably a good match.
After you have done all your self-assessment and homework, Brechner suggests one more test to decide if the move is right: "Go with your gut." Change is sometimes difficult, even when you have studied the situation. If you feel like it is a change that will teach you something new and you feel good about making it, then chances are it is a good one.
This is the end of my first section about tester’s career. Other than the four career paths, we have other different paths as well. For example, we can grow as Test Lead, Test Manager, PM and dev or we can even find a career not in Computer area. As a tester, you have a much board view of the system, you think more on customer than the code, you think hard on why we should develop the feature, how can I make sure the feature is what our customer need. These skill sets you learned from testing can give you some advance for your further career. In the next section, I will discuss several areas our tester can improve to be a better engineer.
Part II Advises
In this part of the article, I will focus on providing suggests for you to grow your career. It is true that changing might take time, and you will become a senior person in a day. Keep studying, keep thinking and grow your own interesting is the key of your career succeed. I hope the article can help you thinking and start to grow your strength. Take me as an example, I used to only focus on the projects I was given and spend less time thinking. At one day, I accidentally visited www.infoq.com, and listen the talks “Testing is Overrated”. After reading the talk, I shared my thought to my teammates, and realized that there is so much great information outside of my work. I started to read these articles and borrow testing books. It took times to have such habit of studying and thinking, but once you built such ability, you will find out you can grow so quickly.
Sometime, people will feel boring about doing the similar thing every day. They start to lose passion, and feel that their career grows becomes slow. How we can handle this situation? I have some advice to you.
- Consider to leave your comfort zone.
Once you stay in a group for long time, you have a comfort zone which makes you feel that your job has less challenge and your skill did not grow. So it is the time to change. You can either change to other group or change to other project which you are not familiar with. Please do seriously consider this because it has importance impact on your career. In the future blogs, I will discuss switching or nor switching in details. In general, I think changing is always good thing which you should always consider. I saw many cases that people switch to other team and lead to better career. Also, consider that switching to other team give you chance to learning new skills which will eventually be benefit for you.
- Consider to do some side project.
My second advise is that considering to doing some side project. What I found during the past years is that the side projects which people build during their free time or out of main responsibility tend to have much larger impact that the funded project. As a professional engineering, we should be self-motivated, and self-organized. If I am doing something fit my interests. I will be very passion on the stuff and tend to make this effort continuously.
- Grow your interests
My third suggest is that try to grow your interest to other area. For example, when I feel boring on daily works, I usually go to http://OfficeTalk to know what happens inside Microsoft Today. I like to read articles at www.infoq.com to know what happens outside of the company. I enjoy reading Google testing blogs about what they are doing. You can choose an area you are particular interesting and have a good behavior to learning new thing every day.
Finally, I have some advises for our manager: Doing macro management instead of micro management; leaving people some freedom to do things; Encouraging people to explore different opportunities. I know we have commitment and we have task to be finished. However, making people happy and be motived is more important that delivering a feature. A happy team deliver better product and we don’t want to be always stressful.
To the end, I am a true believer of Google’s 20% time and our Garage project.
Once you stay in one group for long time, you obtained very deep knowledge about the area and the testing methodology for the area you own. In this case, we tend to be comfortable on what we are doing now, and sometime want avoid change. However, as a professional tester, we should always think broadly, thinking about what new techniques we can use, thinking about what about the future testing technique for your area. In general, a good tester should think ahead of what we current have, and have some plan to adopt changes.
Why? The reason is the technique changes rapidly, if we don’t think ahead and prepare ahead, one day when changes happens, you will find that you are suddenly facing so much challenge. For example, I always read www.infoq.com and www.dbms2.com to enhance my skill. Once our team decides to implement Column Store for data warehouse scenario I already know why we should do this, and what is the hottest technique for this are.
In order to build such skill, we need be open and be broad. We need to know what happen inside the company and what happens in the community. We should be open to listen and learn ideas from others. I strong urge our senior testers closely communicate with members from other team, especially other groups in Microsoft, and build the ability to learn and adopt techniques quickly. At one day, you will feel that the investment of learning will have big return to your work.
I can give you one example of how I achieved this. In my case, I subscribe a couple of very active blogs inside and outside of Microsoft and receive news from others. I also attend talks and training to grow myself. The talk can be very broad, such as service engineering, scenario focus engineer, etc. By attending such training, you will have boarder knowledge about the technique. Also, you know what happen inside the company. I attended two $99 training outside in the past year, and I brought ATDD and Personal Kanban into our team. A couple of team members inside SQL are using the techniques and ATDD was rapidly adopted by many teams inside Microsoft. You can see the value of be open and be broad to help you grow as a senior tester.
Feel free to ping me if you want details about what group and what alias I joined.
Today, I like to talk about another topic of being a senior tester which is making big impact. One of the important way to measure the achievement of a person is how big the impact you are making to the team, to the project and to the customer. I have three areas which you can make big impact.
- Help other's grow
We need realize that no matter how smart you are, you can be not succeeding with yourself only. The more you help others grow, the more likely you will be succeeded. As a senior tester, I always enjoy to see junior tester grow their skill and their career, and I also will to help them grow by providing suggestions and mentoring them. From my heart, I believe that help others is the most important area we should build and applied every day. There are many ways you can help other grow, such as help them to doing project, answer questions in the forum, mentoring new members, teaching them how to coding and how to testing. Building such culture is super important for the org because people will feel warm from others, and encourages sharing and learning. In the end, we grow together as a team.
- Influence others
Once you become more and more senior, you have built up a very deep knowledge about the techniques, and have more experience on doing projects. You get more respect from others and become to the GOTO person for certain area. In other word, you have the ability of influence others. If we looked at the Architects, Techniques Follows and Distinct Engineers, their idea and mind can influence a lot of people and such ability is their unique asset.
Do you think we can influence others like the big guys? I think so. Everyone have certain area which you are the expert. You should use your expert to help people make decision and provide valuable suggestions. For example, for each project I was involved or I learned, I do have unique view of the project, I tried to understand why we should build such project, I think more about why not we use another way to build it, etc. I always share my thought to the people in the project and we make decision together. I wrote lot of blogs to share my thought, and hopefully influence more people.
- Do more cross team collaboration
There is one thing which I think our SQL Server team did not doing well is the cross team collaboration. In the past, we have different testing team for different area, such as Engine, DP, Manageability. However, we don’t usually share best practice, testing tools or ideas together. We only work together once delivering a feature. In this sense, we did not doing cross team collaboration very well. The same is true for SQL server collaborate with other teams. We did not share too many testing tool to others, and we did not use other’s testing idea as well. As a senior tester, we should improve this area by engaging with more and more people. Learning from others and sharing with others can reduce the cost of testing a lot, and we can only grow if we start to learn from each other. So my finally suggestion is that reach out to more people and do more cress team collaboration.
Take me as an example, in the recent years, I brought ATDD into our team, and introduced it to many other teams inside Microsoft, such as Bing, Lync team. I also attend different kind of meetings and seminars to know how other team doing tests. Whenever I saw someone is doing project related to the area I am familiar with, I also ask whether they need help.
In conclusion, when you are making big impact, you are also growing your experience and growing towards becoming a senior tester.
Today, I like to discuss the single most important skill our testers should master in their career, which is coding.
- Why coding is important?
Because you are SDET, Software Development Engineer in Test, you are software engineer. As a software engineer, coding is the task you should do every day, and that is the skill you should get. You may ask whether writing tests is not important than coding. The reason is that writing tests can help the product improve quality, but it does not grow your career sometime. I can give an example of mine. When I first join SQL Server, we write T-SQL script based tests, and I have less chance to write coding. As a result, my coding skill did not grow. Fortunately, SQL Server testing team moved into programming way of writing tests today, and our tester's coding time increased a lot. That is super great. Of course, we sometime spend too many time to write code and write library, and less time to write the real test case. That is another big topic; I am not going to talk in this blog.
Since most of us writing automated tests today, does it mean that we have great chance to improve our coding skill? The answer is not really. Our SDET are doing too many tasks today (just as our PM play different roles): we write test libraries , we do lab run verification, we sign-off product, we setup machines for testing, we install builds for testing, we fix our fragile tests, we open and close bugs, we create enlistments, we builds (radically, we spend lot of time for building). We also have other tasks, such as meeting, project tracking/bug report. All above tasks will take a huge amount of our daily times, and the time reminding for us to doing real coding and grow our skill is pretty small. I remember one day I mentioned that I dream I can spend 50% time on coding to our test manager; he is pretty surprise in that he believes the number should be much higher. However, in reality the number is much slower.
So how can we do for this situation? We should try out best to improve our engineering system to reduce the time for unnecessary times, let system to setup environment, install build, and run tests, open/close bugs and sign-0ff. All of these should be automated. We should commit our self to doing code every day and do coding as much as possible. If you cannot achieve this due to the nature of your job, you SHOULD consider moving on to other job.
To summary, keep in mind coding is your single important skill you should grow.
In recent days, I tried to understand how we should teach or how student learn. My experience of doing Ph.D study and my recent Dale Carnegie Training provide me some thought on this:
- The best way of teaching others or sharing experience to others is to make them thinking. No matter what they thought during your talk, they learned. A good practice is to encourage them to talk and interact with you.
- Thinking itself sometime might not be enough; we might need practice and apply our thought in our work.
For researching papers, most of our paper follows the classic styles, which must have introduction, related works, experimental result, and conclusion. A paper without experimental result is nearly impossible to be published. On the other hand, the essential of a paper is the idea, which somehow was hidden in the paper or not trivial to find. I think that is one of the issues of researching.
The same thing can also apply when we want to do presentation to show people something or we want to write something to teach others. First, you will spend lot of times on researching on what I should think. After you have something idea in mind, you will eager to share with others by writing and which is a good way to summarize your idea.
Finally, I believe the value of senior tester is the idea/skill he/she can bring to the team, not the work he/she did in the past. Being able to think hard problem and find solution is an important skill for our tester and I hope thinking is the biggest skill our senior people need to have, and a great leader must be a great thinker first.
I believe as a senior tester, we should fully understand the product we are testing for. Knowing the direction/future of the production is the first step of creating better tests. In other word, if we don't understand why and what we should build, we will not write good tests.
We should be more involved in project/product planning, and make big impact on the strategy of the product (not only the test strategy). Note, this is one of the important way which we can improve product quality. If we can find design bugs, we can save lot of time and money, and that is even better than finding lot of functionally bugs. Interestingly, I believe a good product design and a right direction can lead to less functional bugs. I was involved a couple of improvements in the past, what I found is that if the feature is well designed, we will see less product issue/bugs/worries during the implementation of the feature. However, if the feature is not well designed or we should not implement the feature, you will see lot of issue during the execution of the feature.
Being involved on production design can also help us enhance skills about managing/build a project. And enhance our technique skills which are crucial for Test Architect and Domain Expert career path.
Understanding the production helps team members to speak the same language. Suppose at one day, you want to join another company to do Cloud Computing, when you talk with the hire mangers, they may ask you a lot of questions about Cloud Computing. If we only know how to test a single component in your service, you will find out you are lack of knowledge/thought, which will impact your future career. However, if you knew and thought about Cloud Computing techniques, such as IASS, PASS, Amazon AWS, etc, I bet you will have better chance to get an offer. This is always true for joining a Startup Company, having a skill other than testing is super important.
Finally, I like to share the thought from Erwin Engelsma
"Testing can improve customer satisfaction if you actually know what the customer thinks is truly important and test for those subjects. Making great improvements to an area that your customer is hardly interested in is a laudable effort, but won‘t much increase their idea of how good your product is!"
-Key Questions to ask when improving testing by Erwin Engelsma.
One day, my manager asked me: "Qingsong, why you can get good review result when you are in SDET II level". At SDET II stage, I don't have deep testing knowledge and don't have big impact on the team. So I wonder what make me having a good review result and the answer is doing things differently.
One way to thinking about this question is that how you can difference you with others. What I found is that when I was assigned some tasks, I did a couple of things in additional to what I should do, and which make me difference with others, and it is the main reason I made bigger impact and grew my career. Here is some example of what I did in the past:
- When we plan to add Date and Time support in SQL Server. I spent lot of time research on Date/Time and Time Zone support in Windows, Linux, .Net and Win32 API. I was actively involved in project direction and design. This made me in a better position when we testing the feature. In additional, I took more responsibility during the execution of the feature, including managing the build, managing the lab run, review books online and help others to write test cases, It grew my knowledge and also help me making big impact.
- After we implemented Sparse Column feature in SQL Server 2008, I did not stop after our feature RIed. I was actively finding ways of using our feature internally. Finally, I found our team's VSTS system can be used for this feature, and I work with the support team to deploy the feature into the system. In this way, I help the org's business and also better understand the user scenario of our feature. As the result, I identify a couple of missing functionality of the feature.
Finally, I hope you get the idea of doing things differently. It will help you a lot for your career if you build such ability.
Today, I like to write a blog about hiring testers. The main reader is our hire Manager. This blog is not about how to interview people or decision on hiring a person or not which I think it is what part. My main topic will be the Why part, i.e., why we need to hire one or more testers.
I am not a test manager, and I don't know what the reason our manager said to HR is when they need more people. Maybe let me put some of the possible reason:
1) We start a new project or feature, and we need to setup a new dev and test team.
2) We have a new test lead, the lead should at least manage 5~8 people.
3) We are short of test resource when doing a project
4) Our VP just gives the openings to test manager, and if we don't fill them, they will be "wasted".
Do we really lack of test resource?
I always heard people saying about lack of test resources for their project. However, do we really lack of testers? Not really, and not at all. There is no lack of test resource issue inside Microsoft, but rather resource allocation issue. Today, our tester usually belong to a component team, lead by a test leader. He/she grow his career by deep understand and testing very well for his area. People tend to believe that for each area/component we own, we need a test team. People tend to believe that testing is a professional which need deep knowledge about testing. We may think about this issue in other way. Today, the modern testing framework, such as NUnit, Xunit, MsTest and Selenium make writing automated test very easy, and do testing don't really need too much knowledge about testing, especially for white box testing (I believe white box testing has more power than black box testing in the sense it was written by dev. and run as early as possible).
I saw a couple of cases that our senior tests have very deep of domain knowledge about their component and for such component, deep understanding is necessary. For example, testing query optimizer is the case. However, I think the best tester should turn his/her knowledge and the testing idea into test library, so that everyone can use it, and which making testing such component much easier. In SQL Server, TestQP and QREL are very good example, they are the tools that have built-in knowledge about query optimizer and relational database. After you transfer your knowledge into code, I think you are free to move to other team, we are not necessary to keep in the person because he/she has the best domain knowledge.
Growing our team does not mean growing our business?
Sometime, people feel proud that a team grows from 5 people to 20 people or even more. However, it does not mean that we grow our business four times. The number of people managed should not be the way to measure the success of the manager or the team.
Do you think adding new testers will improve the team's productivity?
It is not really. It is true that sometime, our testers are very busy on projects and we have a feeling that adding one or more tests can help us? If it is the reason we want to hire people, what about the project finished? We keep the people forever.
Here is some of my recommendation to our hire manager if he/she wants to hire a new tester:
1) Trying to explore different approach to solve the issue of need a tester, and treat hiring a new tester as the last solution.
2) If we need more testers for a project, can we move tester from other team?
3) If we have too many things to do, can we priories them, and cut the lower priory tasks.
4) Thinking about growing a lead to be a technique lead instead of people management lead. We tend to grow very good IC to lead position and let him/her managing more people. However, our lead today spend too many time on people management, and other stuff, they just have no time to think, no time to improve their technique skill. So please consider treating our lead as technique lead, In this way, the number of people who managing is not important, but the people who influenced do.
5) Always spend time to improve our culture, our process and our methodology. Engineering excellent is the key for better productive. Reduce our technique debit, spending time on innovation.
6) Consider to build up some metric to measure the efficiency of tester or our test, so that we can use better way to make decision.
How about the career of new testers?
It is a big topic, which I will not talk too much here. One way to think about is that we all want our people grow fast and have a better career in the future. We all want our tester can easily find a testing job in other company if they decide to pursue opportunities outside the company. However, many companies today have relative low ratio of dev: tester and they proved that their product quality is not bad because of this. I hope someone can do some research on the job market, the level of tester in other companies, so that we can use more fact to understand this issue.
This is the end of my series of blogs of becoming senior testers. I hope you can learn some useful information from my blogs and help you decide your career. In recent years, Computer techniques have been changed rapidly. Cloud computing, social network, and mobile are the hot areas. The change of techniques required different kind of testing techniques as well. I will start to write another series of blogs about the future of testing and tester's career. In the next paragraphs, I will list a couple of latest articles to answer the questions about what is the future of software testing, testing in the service area, and the impact on tester's career.
There are a couple of references related to the future of testing:
- At the Google Test Automation Conference 2011, the opening keynote was presented by Alberto Savoia, Director of Engineering and Innovation Agitator at Google. He believes that software testing as we knew it is dead – or at least moribund – in which case we should stick a fork in it and proactively take it out of its misery for good. I watched this video twice with several colleagues, and we all feel that it is the best talk which letting us to thinking testing and career more seriously. I strong everyone in test role to watch the video at here. The topic mentions that that Startup Companies are interesting on more “we building the right thing?” than ”are we building right? I.e., the quality here is really not whether my software or my service having bugs, but rather is my idea the best idea which can get interesting from customer. This has some impact on our tester since we are too focus on
are we building right” and might cause us harder to find jobs in Startup company.
- Crowdsourcing is a very hot topic recently. Can you image that having a company with tens of thousands testers which can help other company doing testing in a very short time. These part-time testers are paid by the bugs he/she found. They run tests with different devices at different location with different languages. Unlike our in house tests, they run tests as real customer. uTest.com is such company which is super-hot in this area and it will make big impact on the way of testing services and mobile apps. Internally, we have several teams, including Bing, Lync, are actively using Crowdsourcing to testing their feature. What does it mean to our tester is still unknown?
- "How Google Tests Software" by James Whittaker, Jason Arbon and Jeff Carollo is a book that details exactly what is described on the cover. It is an informative and interesting look beneath the covers of how a large technical organization like Google deals with the complexity of software testing. An interesting fact is that just before the book printed, all three authors left Google, one came back to Microsoft becoming a Dev director, and the other two joined uTest.com. Here is one quote from the interview at here:
InfoQ: Throughout the book you give the advice to "not hire too many testers" and that the future shows the Test Engineer role in decline. How do you respond to organisations that would argue you need more of these roles to delineate the line between developers and quality assurance?
Why would you want such a line? Google has proven that when the line between creating code and making that code better is blurred, the result is code that is developed much faster with fewer latent defects. Hiring too many testers creates a crutch for developers that is bad for the product. It annoys me when people relate too strongly with their role. "I am a tester" is an unhealthy attitude. So is "I am a developer." When people stop focusing so much on their role and start focusing on their product, that's when the magic happens. That's when everyone is focused on doing whatever it takes to build the best product they can.
InfoQ: What is the best advice you could offer to current test analysts or new graduates who are considering a role in testing to meet the changing skills of the role?
Treat testing like development. Get a CS degree and get good at CS. Certificates and industry training will only teach you the easy stuff. Learn the hard stuff and get good at it. Testers who take the easy way out will still be griping about being treated as second class citizens until the cows come home. Don't want to be treated that way? Then obtain first class skills.
- Bing team has the principle of Combined Engineering which is very interesting for testing the service and the career of tester. In Combined Engineering, Software development engineers (SDEs) and software developer engineers in Test (SDETs) were combined into one “engineer” role, we optimize for delivering services, not software. In another words, many testers become developer, the true developer who only write code, not testing. I believe this might be the future direction of service team, and today’s tester can focus more on Monitoring, Infrastructure, and Tools, they are just the same as developer.
- Our Testing in Production Expert, Seth Eliot, believes that TestOps is one of the future paths for our tests. You look at this link for related information. I think Testing in Production can really change the way of how we do testing and the future of tester’s career. Here is one section in the interview about TestOps
It’s more of an evolution of what I have already been talking about around testing services and Testing in Production. I call it TestOps.
Testers need to move out of the mindset of writing tests – running tests – evaluating results. Instead of using test results from a daily run as your quality signal, use the big data pipe coming off of your product (generally this refers to services) as your quality signal. This includes system data like CPU, API requests, system response time as well as (properly anonymized) user data. It also includes data emitted from synthetic transactions that you can run constantly in production. These are indeed test cases, but instead of getting just a daily red/green, you get constant availability and performance. This is the technology, but it also necessarily will change our software engineering organizations. The questions of roles and specialization versus generalization will need to be answered to meet each organization’s needs. And the emergence of the Data Scientist as part of the engineering team is an exciting outcome of the TestOps approach.
There are a couple of references related to our careers: