How to use open-source intelligence to get to the truth

Analysis and Commentary, Emerging Technologies, Featured, Ploughshares Monitor

Published in The Ploughshares Monitor Volume 43 Issue 2 Summer 2022

On April 6, the Peace and Conflict Studies Association of Canada (PACS-Can) presented a forum on open-source intelligence (OSINT) that featured Ploughshares Researcher Kelsey Gallagher, with Ploughshares Senior Researcher Dr. Branka Marijan as moderator.

Some days later, I saw a feature on the same topic on the PBS NewsHour. Apparently, OSINT not only serves as a source for news and other media, but is itself considered newsworthy these days.

The term “open-source intelligence” refers to data that is accessible to everyone. At one time, this would mean sources that could be readily found in public and university libraries, in newspapers, books, journals, government documents, and curated collections.

Mostly in hard copy, with some microfilm/fiche. But with the rise of the Internet and the ubiquity of social media, OSINT now often refers to posts of all sorts from “citizen journalists” – indeed, from anyone with an account and a device. Some, but not all, of this information is valuable and is becoming more and more critical for modern researchers.

Trusting in partners

One of Kelsey’s primary responsibilities at Project Ploughshares is to track exports of Canadian military goods. In some cases, this includes learning how the goods are used after export and if such use matches up with the intentions expressed by the importing state.

By assembling data from various sources, he might learn, for example, that a piece of equipment, such as a sensor, which was exported for use by one country is being used by another – an example of diversion. Some of this information might come from governments and manufacturers, but a lot is from social media.

In the end, Kelsey might be able to claim that certain weapons are being used inappropriately. In September 2020, for example, Ploughshares published a major report authored by Kelsey that indicated that the transfer of Canadian-made L3Harris WESCAM surveillance and targeting sensors to Turkey “poses a substantial risk of facilitating human suffering, including violations of human rights and international humanitarian law.” The report garnered a lot of media attention and was at least partially responsible for a federal government investigation into the export of these sensors to Turkey. (The report can be found on the Ploughshares website. Look for Killer Optics: Exports of WESCAM sensors to Turkey – a litmus test of Canada’s compliance with the Arms Trade Treaty.)
But how do researchers and analysts reach a sufficient level of certainty about the information found on social media platforms like Facebook, Twitter, or Instagram? Even accredited journalists and professional academics, whose work goes through a vetting process, can be subject to political or other pressures that lead to biased or even false content. The bar is much lower for information posted by observers and enthusiasts, which can be accidentally or intentionally inaccurate or just plain false.

Kelsey knows that he can trust the process of certain sources, with whom he has had extensive contact over time. Canadian researcher Anthony Fenton, a PhD candidate at York University in Toronto, is one such contact; he has been critical in assembling information on how the Saudis use Canadian weapons. (See Q&A: Mining social media for peace.)

In the PACS-Can discussion, Kelsey spoke highly of the Netherlands-based group Bellingcat, “an independent international collective of researchers, investigators and citizen journalists using open source and social media investigation.” It not only investigates a range of subjects, but offers training in how to do the type of work they do. He also spoke of PAX, another group based in the Netherlands. Both are trusted partners.

Not all useful sources share our focus on building peace. Some are weapons enthusiasts who track certain weapons from source to destination. They visit black markets, see what’s on offer, and then make that information available for a fee. On occasion, Canadian-made weapons, including firearms such as carbines, are found, providing evidence of diversion that wouldn’t otherwise be available through conventional channels.

Verifying data

Ultimately, however, Kelsey makes sure to do his own due diligence before accepting information as fact. During the forum, he offered some tips on how to do this:

  • Make certain that more than one source can confirm evidence that might be suspect.
  • Seek out video evidence rather than photographs, which are easy to fake.
  • Do your homework so that you are less likely to be tricked by fakes or false identifications. Learn all you can about the subject of your investigation – a particular weapon system, for example. Know what similar systems look like and know how to distinguish key differences.
  • Learn enough about relevant contexts to judge the likelihood that the data under examination is accurate.

Kelsey referred to images published by Yemeni and Turkish government agencies that show their armed forces using identifiable weapons. He saw no reason to discount such sources. Random images, however, could easily be fakes.

Ultimately, as Kelsey has noted, “open-source imagery is only one aspect of verifying weapons exports. It is frequently used in conjunction with other data.” For example, a Canadian government report on exports could indicate that military goods have been exported to a state that becomes involved in an armed conflict. Subsequently, Kelsey could find images of what appear to be Canadian weapons taken at that conflict.

The ethics of OSINT

The PACS-Can forum also discussed the ethics of OSINT, particularly the publication of images that will be seen as personal by some viewers. Many respected organizations, including Amnesty International and the United Nations, use open-source images. The basic operating principle in such use is to do no harm. But who gets to determine if an image causes harm?

In most cases, no one has given permission to use the images of the dead and injured. And what about the relatives of those pictured? Would anyone want to learn that a loved one had been killed or injured by stumbling across an image on a website?

Another, different case involves photographs of prisoners of war, even those who are well treated. It is never permissible to publish such photos.

Proceeding with caution

The sheer volume of open-source material is a major concern. At one point, Project Ploughshares considered using artificial intelligence to keep track of shipments of armoured vehicles. But it turned out to be both difficult and expensive to train an algorithm to detect a particular armoured vehicle, which can look a lot like a different armoured vehicle or even a tank. The result was a lot of false positives and probably just as much work for humans checking on the work done by the machine.

There is no central OSINT repository. Most researchers are working on fairly narrow topics, like cluster munitions in Ukraine. And so a lot of cross-checking must still be done.

It is certain that Kelsey and other researchers will continue to use OSINT. But it is critical that all such use be done with due concern for ethical considerations and respect for truth. This means keeping up with the tech, for sure, but it also means holding fast to basic principles and integrity.

Spread the Word