OSINT: Sources and methods

In the modern world, information plays a key role in many aspects of life, be it business, security, journalism or government. OSINT — Open Source Intelligence — is one of the methods for obtaining information, including collection and analysis of data available from open sources, such as the Internet, media, pubic databases, and social media. Below, we will explore what OSINT is, what methods and tools are used for intelligence collection, will give examples of using it in various settings, and discuss whether it makes sense to learn how to gather OSINT and search for information on your own.

OSINT is intelligence produced by searching for information in open sources. It is the process of collecting, analyzing and interpreting data from publicly available sources aimed at obtaining useful intelligence.  OSINT investigations are relevant in various contexts, including cybersecurity, business intelligence, journalism, law enforcement, and even national security.

Some of the main purposes of OSINT include the following:

• Detect vulnerabilities. Finding weaknesses in systems and applications.
• Monitor data leaks. Search for disclosed data and accounts on the Internet. 
• Prevent attacks. Identifying potential threats and preparing for preventing or eliminating them.
• Obtain critical data. Collecting and analyzing information for personal use and informed decision-making.

OSINT uses a wide range of sources that can be conventionally divided into several categories.

1. Internet:

• Websites: Public and private webpages.
• Social media: User profiles and posts.
• Forums and blogs: Discussions and articles.
• Search engines: Data search by keywords.

2. Media:

• News: Articles, reports, and press releases.
• TV and radio: Broadcasts and podcasts.
• Magazines and newspapers: Publications in printed media.

3. Public databases:

• Government registers: Registration data of companies, land cadastres, etc.
• Scientific publications: Research papers and reports.
• Commercial databases: Marketing research and reports.

4. Reports:

• Government: Public reports and documents.
• Commercial: Financial statements and presentations.

OSINT search methods are quite diverse. Some of the most common and popular are discussed below. Firstly, it is web scraping. This means automated collection of data from websites that makes online research much faster and more efficient compared to manual search.

Next, it is social media analysis. The importance and benefits of exploring user profiles and activity on social media cannot be overstated.  Now, they accumulate not only personal data, but also important business information and real reviews, criticism, analytical overviews, and other significant materials.

We cannot also forget about news monitoring — tracking news publications and events helps produce a fair assessment of the current situation and make correct forecasts.

OSINT business intelligence

 Purposes 

• Competition analysis. Collecting information about competitors, their products and strategies. OSINT competitive intelligence is important at all stages of a business existence. It is very difficult to do it efficiently on your own — you need to have appropriate skills and knowledge of the methodology. 
• Market research. Analysis of market trends and consumer preferences.
• Investment intelligence. Evaluation of promising companies and startups for investment. 

 Methods and tools

• News and publications monitoring. Tracking news and reports about competitors.
• Social media. Analyzing profiles and activity of key persons in competitor companies.
• Commercial databases. Using databases, such as Bloomberg or Hoovers.

 Examples                                                                                                                                 

• Financial statement analysis. Using publicly available financial statements of competitors to assess their financial position.
• Social media. Monitoring posts and announcements made by competitors on social media to understand their marketing strategies.

OSINT in journalism

 Purposes 

• Investigative journalism. Using OSINT to investigate corrupt practices and other crimes or other socially significant incidents.
• Fact checking. Checking information for accuracy.
• Event monitoring. Tracking current news and incidents.

 Methods and tools

• Search engines and news aggregators. Using Google News, Factiva and other tools to search for news.
• Social media. Analyzing publications on X, Facebook and other platforms.
• Documents and reports. Studying information from public reports and documents.

 Examples                                                                                                                                 

• Investigating corruption. Collecting and analyzing data about government contracts and tenders to find signs of corruption.
• Fact checking. Checking statements made by politicians or companies using official reports and data.

OSINT in law enforcement

 Purposes 

• Investigation of crimes. Collecting evidence and information about suspects.
• Social media monitoring. Tracking activity of criminal groups.
• Search for missing people. Using open data sources to find missing people.

 Methods and tools

• Social media analysis. Exploring profiles and posts of suspects.
• Web scraping. Collecting data on forums and websites.
• Exploring public databases. Using government registers and databases.

 Examples                                                                                                                                 

• Monitoring of criminal groups. Tracking activities of suspicious groups on social media.
• Evidence collection. Using public records and documents to support facts in investigations. 

OSINT in national security 

 Purposes 

• Threat monitoring. Tracking potential threats to national security.
• Risk assessment. Analyzing risks and preparing for possible incidents.
• Counterintelligence. Collecting information about activities of foreign intelligence agencies.

 Methods and tools

• News sources. Monitoring international news and events.
• Social media. Analyzing posts and activity of users.
• Specialized databases. Using data from intelligence agencies and international organizations.

 Examples                                                                                                                                 

• Terrorist activity monitoring. Tracking posts and messages on social media related to terrorist groups.
• Political risk assessment. Analyzing international news and reports to assess the political situation in different countries.

Sometimes, performing open source intelligence gathering on your own can be a good idea for obtaining some useful information. But it also has a number of serious shortcomings and limitations that are important to be taken into account. Let us consider the major of them.

1. Limited resources and time

Collecting OSINT on your own requires significant resources, including time and access to specialized tools and databases. Individuals and small companies may not have enough resources to buy licenses for professional tools and subscriptions for databases.

OSINT requires considerable time to collect, analyze and interpret data. Often, information is available in large volumes and processing it may take longer than expected, especially if you have no automated tools.

2. Limited skills and experience

High-quality OSINT training is a must for being able to effectively use it. This effort requires certain technical skills. An OSINT researcher needs to be able to use search engines, analyze data from social media, and know how to use AI for OSINT investigations. By far not everyone has necessary expertise for these tasks. Taking a random training course will not be enough in this case.

Apart from technical skills, to be successful in OSINT, you need to have analytical abilities in order to interpret the collected data correctly. Misinterpretation of facts may lead to erroneous conclusions and wrong decisions.

3. Limited access to information

Many important data cannot be obtained from open sources — you need to have access to specialized or closed databases. Independent intelligence research is limited to what is available publicly but this is often not enough.

Information in open sources may be outdated or inaccurate. Outdated data often lead to wrong conclusions and incorrect actions.

4. Risk of breach of confidentiality and legality

Making an OSINT research on your own may result in unauthorized access to personal data, which may be treated as a breach of confidentiality and ethical standards. Inability to distinguish between acceptable and unacceptable methods of data collection often jeopardizes legal safety of a person doing a research.

Many countries have strict laws governing collection and usage of data. Being unaware of these laws or failure to comply with them can have legal implications, including fines and criminal prosecution.

5. Low quality and unreliable information

Information from open sources may be inaccurate or incomplete. This is due to fake news, disinformation and bias of sources.

Checking information collected for validity and accuracy may be a big challenge. Doing analysis on your own without professional skills and tools often results in errors in the assessment of reliability of sources.

Thus, OSINT is a powerful tool for the collection and analysis of information from open sources. It has a lot of applications, from cybersecurity through journalism to national security. However, when using OSINT it is important to observe ethical and legal standards in order to avoid breaches of confidentiality or law.

In the context of the growing amount of information and constantly changing environment, OSINT skills become increasingly relevant and sought after. But acquiring them on your own is a time-consuming and challenging task almost for anyone. Therefore, if you want to set a course for possessing accurate information about your customers, business partners or competitors, experts of SEVEN SENSES are always willing to provide you with all necessary assistance by performing a relevant audit.