Our Blog

Topic Facets: Continuous Testing

April 5, 2018 - influencer content marketing, Practitioner Marketing -
By: Turner Sblendorio, Chris Riley, Yolanda Fintschenko

When we evaluate our customers’ greatest needs, we always come back to the challenges they face when developing topics for their blogs and assets. While some of our customers may choose to create their own topic strategy, normally, part of our partnership with our customers involves helping them generate a topic for each piece of practitioner-written content we deliver to them. So how do we get at topics? Let’s reveal the magic behind another conversation important to many DevOps tool vendors out there, and get to the nitty gritty of continuous testing topics.

Continuous testing is an approach to improving the efficiency of the feedback loop to get information to developers from development environment-triggered tests as well as more traditional developer/tester-triggered tests. Continuous testing is the process of executing automated tests as part of the software delivery pipeline to obtain immediate feedback on the business risks associated with a software release candidate.  As the software development philosophy has shifted from waterfall to embrace concepts such as Agile and DevOps, the need for automation for continuous testing has grown.

Google Trends Results for Continuous Testing for February 2018


While many approaches to determine the share of conversation that a company will need to capture rely on predefining the competitors for share of voice, we use our calculation to determine who the competitors for share of conversation are in continuous testing. Tools like Google Trends help us see the larger space, while we use Re:each to define the microenvironment.

Our approach to determining topics within this conversation begins and ends with a share of voice (SoV) calculation, which ultimately is giving us an idea of a vendor’s share of this conversation (SoC). Our share of voice methodology is described in some detail in a variety of places, but here is a quick summary:

Share of conversation is the percentage of any specific conversation you own. SoC is more precise because it looks at specific conversations within a market versus focusing only on global SoV compared to competitors. While it’s interesting to know how your brand or product is doing in the world of all products, you can make the greatest impact by going local with specific topic areas.

Fixate’s Re:each platform has algorithms which derive conversation share of voice across traditional and social media. The phases of calculation are data collection, normalization, and interpretation. We can’t give you the secret sauce, but this will give you an idea of how we do it.

Core Calculations

  1. Identify your place: Identify specific keywords and concepts associated with your brand and product based on those concepts that appear the most in all conversations you participate in.
  2. Determine your conversations: From there, the concepts are applied across a body of sources in order to identify the three conversations which are most relevant to you. For each vendor, there are three types of conversations identified:
    • Market
    • Demand Gen
    • Mindshare/Thought Leadership
  1. Find your competition: Competition is derived by identifying the top 4-9 vendors in each conversation based on their SoV in those conversations.
  2. Determining relevant topics: Topic suggestions are derived from entity/concept extraction of content that was most prevalent in each conversation selected over the set period of time. Those concepts that had the greatest reach in that conversation are weighted and end up as the core elements of a suggestion.

Data is collected from traditional social media sources as well as trusted media sources for each broad market. Weight is put on content based on the source it came from using a proprietary algorithm. Currently, calculations are done at the end of each month for the entire month’s worth of data.

Domain Expertise

The machine learning used in SoV is human-supervised (Human-In-The-Loop). SoV calculations can be fully automated; however, topic suggestions are subject to language challenges, and domain expertise based on raw data collection. Domain experts validate SoV calculations, and reformulate raw entity extraction on top-performing content in each conversation to build coherent topic suggestions.

Results that Influence Topic Selection for Continuous Testing (Feb 2018)

Social Media

Social media posts about continuous testing in February were primarily retweets of the industry news stories put out by corporate accounts or influencers. Good use of CA’s own employees has led to many internal retweets that have contributed to CA’s  share of conversation.

Highest-Impact Industry News

In industry news, conversations have been significantly influenced by a number of announcements and events. News from SauceCon 2018 has had heavy sway, as well as the recent results from Gartner’s Critical Capabilities for Software Test Automation report.

  • Sauce Labs and LogiGear announced that they would be Specialized Service Partners and would be in attendance at SauceCon 2018.
  • Sauce Labs also wrapped up its annual conference (SauceCon 2018).
  • CA Technologies received the highest overall score on Gartner’s Critical Capabilities for Software Test Automation.
  • Tricentis received Gartner’s highest product score in Enterprise end-to-end testing.
  • CA technologies released a study on the best Agile practices. That study can be found here.

Blogs of Interest

Of course, company websites promote their company news. However, it is a post written by Fixate IO practitioner Scott Fitzpatrick, discussing shift-left continuous testing (and hosted on the Sauce Labs website) that went viral.

Practitioner Profile

Continuous testing is another concept that represents the evolution of an existing role. Prior to the DevOps movement, Quality Assurance (QA) was considered an insurance policy, not a strategy. As a result, QA team members had a discrete task to complete, and were not incorporated into the strategy of how applications were delivered. But it became clear very quickly that as applications are released more frequently and faster that the purely tactical approach of finding bugs was not going to work. That is when QA was up-leveled to Quality Engineering (QE)—a more strategic and technical role. This is the key persona for continuous testing—the quality engineer who has visibility and say into how software pipelines are architected.