10 tips for running successful developer surveys
Photo by Jon Tyson on Unsplash
One of the more enduring misconceptions perpetuated within DevRel circles is Developers’ hatred of marketing.
If that statement were true then many of the typical day-to-day DevRel activities would not exist. And besides, have you ever met anyone that likes being marketed to?
Perhaps a more accurate statement is that Developers have a low tolerance for clumsy ill-informed attempts to engage them, which demonstrate a lack of understanding of what is important to them.
So how do you determine what is important to your developers? In this article, we examine surveys, one of the tools you can use to better understand your community.
Surveys? The naysayers will tell you that developers don’t fill in surveys. If that statement were true, then how did 80,000 developers contribute to Stack Overflow’s 2021 Developer Survey?
Below we share our top 10 tips on how to make your developer survey’s more effective.
First, let’s review why you might want to survey developers:
You might be launching a new Developer product and want insights from potential users to inform your product development requirements, value proposition, and product messaging.
You have an existing product and want feedback on its strengths and weaknesses to inform your product roadmap, and its competitive positioning.
You want to take a temperature check on the satisfaction levels within your Developer Community - are they happy with your DevRel Program, do they feel they have a voice, do they feel valued, can they contribute in a meaningful way?
You want to build thought leadership as part of your content marketing strategy. Many companies provide free research reports containing insights and data as part of their lead generation strategy.
Shockingly, 72% of product or service innovations fail to deliver on customer expectations. This failure rate underlines why speaking to prospective users and your existing developers on a regular basis is so important.
So with that low tolerance level for poorly executed marketing in mind, let's explore our tips on running successful developer surveys:
Have clarity on what result(s) you want to achieve from the survey before you embark on a pointless exercise. Avoid conducting a survey as just a box ticking exercise. You only have a limited number of excuses to contact people before you start to become annoying - don’t waste these precious opportunities.
Strive to make your surveys as short as possible. Less is definitely more. Having clarity of purpose will help you streamline the survey to only capture information relevant to your goal.
To help keep surveys short, avoid asking questions you can easily find the answers to from the existing data you hold on your customers. For example - where they are located geographically, which of your products they use, how much money they spend with you? When you ask these types of “lazy” questions you risk: a) showing a lack of respect for your community’s time or b) you may unintentionally surface data collection and management issues to your community. This could influence their opinion of your technical competence.
If you are surveying customer satisfaction, define a set of core questions and repeat them each in each iteration of the survey so you can compare results over time and measure trends - are you doing a better/worse job?
Consider using an industry standard like Net Promoter Score (NPS) so that you have a recognized metric that allows benchmarking with competitors or other industries.
Make your surveys digital so there is no paperwork to complete and less manual analysis and presentation of results required.
Create a culture that seeks excellence. Consider every contact with a customer or prospective customer as an opportunity to gather insights and feedback. This might be a support ticket, chatting at an event, a post in a forum or social media, etc. Ensure your team is trained to spot these opportunities and have the tools to capture and report their discoveries back into the organization. This culture permeates via leadership, training, hiring criteria, and your day-to-day activities. It also shows developers that you care and are listening.
Create a culture of transparency. Don’t fall into the trap of cherry picking survey results that paint you in a positive light while suppressing the negative stuff. Publish results. This shows that you have nothing to hide and are striving to improve. If you are doing well, this contributes to your competitive advantage, and if you are struggling it demonstrates your desire to improve, and may encourage developers to help you. People love a scrappy underdog. Developers value honesty.
Ask what will change once you have received the survey results? How does product feedback get assessed and prioritized onto the product roadmap? Who in the DevRel organization owns customer (developer) satisfaction? Was there a target level you were trying to reach? Did you achieve it? Why? Establish clear internal ownership, combined with a culture of seeking excellence and transparency. Ownership ensures focus to improve.
ACT. The single most powerful way to improve your engagement with developers is to have a demonstrable track record of acting on and implementing their feedback. This might be implementing new product features, bug fixes, tweaks to documentation, a new library, adjusting pricing or policy changes, running an event in a new city, the list goes on. Even if there is a valid reason not to implement a popular suggestion, acknowledge and thank them for their contribution, and if possible indicate why you are unable to implement it. Everyone likes to be heard and valued.