Market intelligence

This article is about information and analysis of a commercial market. For information to support marketing activities, see Marketing intelligence.

Market intelligence (MI) is gathering and analyzing information relevant to a company's market - trends, competitor and customer (existing, lost and targeted) monitoring. It is a subtype of competitive intelligence (CI), which is data and information gathered by companies that provide continuous insight into market trends such as competitors' and customers' values and preferences.

MI along with the marketing capabilities of an organization provides a guideline into the allocation and implementation of resources and processes. It is used for the purpose of continuously supplying strategic marketing planning for organizations to gauge marketing positions in order for companies to gain competitive advantage and best meet objectives.

Organizations can develop MI frameworks and models that are suited to financial capabilities and desired market sectors but are mainly based on the four-step process of collection, validation, processing and communication of MI. The gathering of MI data is sorted into many different categories, including, but not limited to, qualitative, quantitative, formal, informal, published, and unpublished. MI data is gathered both internally and externally.

Benefits that MI can bring are that it provides customer, competitor and market insights allowing organizations to gain a competitive advantage in their marketing strategies. Issues that MI can bring is through acquiring data and information through illegal or unethical ways, it can lead to financial loss and government regulatory failures.

Background and Development

MI and its broader term, marketing intelligence, was first introduced in “Marketing Intelligence for Top Management” by Kelley, to provide information that was analyzed, reliable and consistent for an organization to better create policies and make business decisions.

Following Kelley, in “How to Develop a Marketing Intelligence System”, R. Pinkerton shows the proactiveness of organizations as marketing intelligence systems is applied whilst the technological revolution arises. Contributions to MI include professional organizations such as “Global Intelligence Alliance” and “the Society of Competitive Intelligence Professionals” (SCIP). These organizations have contributed both empirical and theoretical research in an attempt to further define and understand MI.

As research into MI comes from scholars and non-scholars of different backgrounds it has resulted in a fragmented state of research. This has led to MI being used interchangeably with other market terms such as competitive intelligence, business intelligence and strategic intelligence. MI to this current date continues to change to meet organizational requirements.

Framework

The implementation of MI varies depending on how organizations perceive it. MI is defined as being composed of three main activities, these activities are Information Acquisition, the gathering of marketing information that is required for current and future customer needs, Information Analysis which is the intelligence gained from the information collected and Information Activation, which is using the intelligence to implement and develop marketing plans.

Frameworks can be flexible, however the basis that organizations use to model the MI surrounds a four-step process, which are, collection, validation, processing and communication. Data mining techniques are used throughout the processes to aid in the gathering and analyzing of data and information retrieved. MI is a continuous process that organizations need to keep track of to improve their strategic and tactical marketing planning. These processes target the three activities that MI is defined by. The model can be adjusted and adapted when required and can be implemented all at once or by sections.

Collection

Collection is the first step in the MI model, it involves the gathering of data and information of a particular market sector. Such data and information can be gathered from external sources, such as other organisations and their market strategies, research institutes and business reports.

Internal factors can include looking into current strategy processes and personal customer trends. It is estimated that 70% to 80% of intelligence resides within organizations employees or, internal MI network, as they are the team who gains information's when interacting with suppliers, customers and other industry contacts. To involve employees into an intelligence program to gain data and information the following considerations can be noted: developing a rewards program to promote participation, providing MI goals, requirements and a timeframe for information to be given in and creating a proper communication method to promote the intelligence program with employees such as using an e-mail system.

A challenge that arises in the collection of data and information is the identification of relevant information, this is a result of organizations not clearly defining a market sector.

Validation

Shows the duplication of data which reduces data quality

Validation is the second step in the MI model, this which can be referred to as data cleansing. The maintenance of good data quality is important as data and information is being retrieved from many different sources. Data and information obtained from sources can be dirty, meaning that it is incomplete, wrong, inappropriate, duplicated. This step will allow data and information to be adjusted and understandable to the organization, furthermore it allows for consistency and compliance to be present. If data quality is not maintained correctly it can lead to organizational losses with revenue and governmental regulation failures.

Method of Validation

Data cleansing is a complex process that involves several stages in order to get good data quality for MI strategy use. Stages include defining the organization's level of data quality, detecting error from the data collected and then repairing the errors. The five stages of data cleansing are data analysis to identify errors, eliminating the errors, checking to ensure elimination of error are done appropriately, refreshing the data in the data warehouse and finally replacing the dirty data with clean data.

Processing

Processing is the third step in the MI model. It involves the use of translating the clean data using organizational rules, modelling, logic and analysis to produce readable information, reports and spreadsheets that allows the organization to gain specified knowledge. The interpretation of data into readable information is difficult as it is complex, it requires proper technology and heavy commitment from a top organizational level to match data and information gained and align it to a marketing strategy.

Communication

Communication is the last step in the MI model. It involves the sharing, delivering and transmission of information gained from the processing step to figures in the organization who will apply it accordingly to the market strategy. As MI is a continuously changing, the communication of the MI strategy requires managers whom have expertise in the given market industry in order to determine the ongoing validity of the MI strategy and its implementation. In order to make the communication of the MI strategy as successful as possible, this process must be performed by every level of an organization, also known as the intelligence organization.

Intelligence Organization

Intelligence organization refers to the “people and information resources who make the market intelligence process happen”. The five elements of an intelligence organization are, MI leadership who manages and leads the MI process, a MI team, a portfolio of external information sources that is set up by the MI team, internal MI network made up of MI users and the MI user's personal information source network. An intelligence organization element is made up of external and internal factors that allows for a continuous MI process.

Gathering market intelligence data

Use of search engines in gathering MI

The gathering of MI data is different dependent on an organization's financial capabilities. Sources of data and information is separated into qualitative, quantitative, formal, informal, published and unpublished. With such sources being retrieved both internally and externally from the organization. It involves using search engines and corporate web sites to see competitor's strategies, identifying business trends through reputable publications and existing customer clientele. Organizations use different systems to gather MI, one system is that is used is Open-source Intelligence system.

Internal intelligence gathering

Sources of internal intelligence gathering include but are not limited to, gathering data from customers, manufacturers, through research and development (R&D), employees, also known as salesforce, physical evidence, sales quotes, sales records, trade shows and new hires. These data sources were ranked by organizations on a scale measuring five for being very important to one being not important. It was founded that customers and manufacturers and R&D are the most important to organizations with one hundred percent of organizations ranking these data sources with the number four or higher^,. It shows that in the process of collection and gathering MI data and information, these data sources brought the most value to organizations.

External intelligence gathering

Sources of external intelligence gathering is included but is not limited to, gathering data from client meetings, dealers/distributors, customers, business associates, market research projects, suppliers, online services, periodicals and government publications. These data sources were compared on the same scale as internal intelligence gathering sources, with results showing that intelligence gathered through client meetings being the most important to organizations, with one hundred percent of organizations ranking this data source with the number four or higher.

Information systems intelligence gathering

Marketing information systems allow for organizations to continuously acquire, generate, and maintain external and internal information. They are systems that make use of artificial intelligence (AI) technology to aid in the planning of strategic and tactical marketing strategy of MI but also share marketing expertise.

Open-source Intelligence (OSINT)

Open-source intelligence is a predominant form of MI gathering that organizations employ. OSINT is defined as the scanning, finding, gathering, exploitation, validation, analysis, and sharing with intelligence-seeking clients of publicly available print and digital/electronic data from unclassified, non-secret, and grey literature. It is frequently used as its system is user friendly, its inexpensive and that it processes an abundant amount of raw materials that can be further processed.

Impacts of market intelligence

Using MI can bring to organizations both benefits and issues depending on how MI is acquired, maintained, and implemented. Benefits that MI can bring includes but is not limited to gaining competitive advantage in their marketing strategies. Issues that MI can bring can include but is not limited to, financial losses and government regulatory failures.

Issues

There are issues that arises in the process of acquiring MI data and information and the implementation of an organizations marketing strategy. Issues such as the acquiring intelligence unethically and illegally can lead to failures with government regulations, also, if dirty data is not properly cleansed and problems aren't mitigated or resolved can lead to a range of negative impacts that can result in financial and reputational losses to the organization.

Legality and ethics

British Airways aeroplane

An issue that can arise is the unethical and illegal collection of data and information. Organizations can collect data for MI illegally or unethically to try to gain competitive advantages; this is known as industrial espionage. An example of illegal MI collection practice is when British Airways breached the Data Protect Act 1984 through accessing Virgin's confidential flight details.

A standard of conduct was developed by the non-for-profit organization Society of Competitive Intelligence Professionals, creating a code of ethics that can be adhered to by organizations when collecting market intelligence, to prevent the illegal and unethical collection of data and information.

Dirty Data

Dirty data that is collected needs to be cleansed to maintain good data quality. Challenges that arise in data cleansing is that there is a large volume of data being received leading to organizations being faced with many risks of failure to detect dirty data being processed through. If data quality is not managed properly, it can result in financial losses, inefficient implementation of MI strategies and failure to comply with government regulations. A reason for financial loss is due operational costs, as there is an increase in resources and time spent to identify and fix the dirty data.

Benefits

MI processes have been used in many organization's strategic market planning, however, there are still difficulties in what the hard and soft benefits in using a MI process for an organization. The benefits of a successful MI process can be sectioned into three categories, better and faster decisions, time and cost savings and organizational learning and new ideas, however, overall, it can improve profitability and the competitiveness of an organization. The competitiveness of an organization increases as with more MI gathered it'll provide a way for organizations to innovate through improving current methods and increasing the ability to find and create new products.