Blogs

Entity Recognition and how it can help LEAs

— by Duszynska-Trojanowska, Anna — last modified Jun 23, 2022 01:51 PM

Everyday language involves extensively talking about things and places associated to the speakers. While these seem trivial in normal scenarios, the places and people talked about can become exponentially interesting to Law Enforcement Authorities (LEAs) in a criminal scenario.

Voiceprints and their properties

— by Duszynska-Trojanowska, Anna — last modified Apr 29, 2022 11:06 AM

Speaker recognition is a technology that uses computer algorithms to analyze speech patterns and determine the identity of the speaker in a recording. Speaker recognition is an important part of the ROXANNE platform because the identities of speakers in recordings from criminal investigations are usually not known. In state of the art speaker recognition systems, recordings of variable durations are converted to fixed sized vectors [1,2,3], often referred to as voiceprints or speaker embeddings. Given such voiceprints from two recordings, their similarity can be measured to estimate how likely it is that the speaker in both the recordings is the same person. In this post we explain how voiceprints are extracted from audio. We also discuss some of their properties and what information they contain.

Location traces in criminal investigations

— by Duszynska-Trojanowska, Anna — last modified Mar 30, 2022 11:29 AM

When reconstructing the events that led up to a crime, establishing the location of a suspect is often of crucial relevance. A suspect’s whereabouts can be informed by tactical and forensic information , and one of the most widely used sources in investigations are mobile phone records that can be provided by telephone network operators. However, these data are hard to analyze and easily misinterpreted. As such, location analysis is part of ROXANNE, a platform for analyzing speech and communication traces.

Privacy-by-Design in ROXANNE technologies

— by Duszynska-Trojanowska, Anna — last modified Feb 28, 2022 01:39 PM

Privacy is an important ethical, social, and legal issue. Yet, some infringement on a person’s privacy by law enforcement agencies (LEAs) might need to take place for effective criminal investigations; this is generally seen as socially acceptable and in compliance with human rights law when it is necessary, proportionate, and done according to legal rules. As such, there are limitations on LEAs processing data about people’s private lives that need to be respected, and the way we design processing technologies is part of that respect.

The assessment of Roxanne Network’s effectiveness

— by Duszynska-Trojanowska, Anna — last modified Jan 26, 2022 01:38 PM

For the purpose of Roxanne project, it is important that the tools and analysis developed by the researchers enable a precise reconstruction of the criminal networks. Exactly for this reason, Roxanne includes also a specific task to assess the contribution of the technologies developed by the consortium to the understanding and analysis of criminal networks.

Vocabulary and Language Model Adaptation for Automatic Speech Recognition

— by Duszynska-Trojanowska, Anna — last modified Dec 21, 2021 08:30 AM

An automatic speech recognition (ASR) system is typically a statistical system using a fixed vocabulary. This means that a word which doesn’t exist in the system’s vocabulary can never be recognized correctly. These words are referred to as out-of-vocabulary words (OOV) and form a major source of errors for ASR. In order to keep the ASR system up-to-date and to decrease the OOV errors, the vocabulary and the language model must be adapted on a regular basis. The language model adaptation component, which will soon be part of the ROXANNE solution, tackles this problem by giving the end-users the opportunity to introduce new words into the vocabulary and to build custom models in a semi-automatic way.

Towards a trustworthy ROXANNE platform

— by Duszynska-Trojanowska, Anna — last modified Dec 01, 2021 12:00 PM

Funded under the Horizon2020 programme supporting ground-breaking research and advancing European excellence, ROXANNE aims to support law enforcement authorities (LEAs) fight crime and terrorism by facilitating the analysis of criminal data. To this end, the ROXANNE platform combines innovative data analysis capabilities, including speech and language technologies, visual analysis and network analysis, to help identify perpetrators. ROXANNE’s innovation lies in the bi-directional interaction between the multimodal technological processes integrated in the platform. The analysis process further benefits from prior knowledge available to investigators for increased accuracy results that contribute to advancing the case. With Artificial Intelligence (Al) at the core of the ROXANNE platform through the underlying algorithmic models, the project team adopted ethics and privacy by design approach in its research work in order to develop an ethically, legally and socially sound final result.

The intelligence cycle and the ROXANNE platform

— by Duszynska-Trojanowska, Anna — last modified Feb 17, 2022 11:25 AM

During investigations the law enforcement practitioners working on the case need to collect and review evidence and intelligence to reconstruct the committed crime or, in case of an ongoing offense, to understand the “modus operandi” of a criminal group and prosecute its members. This blog posts gives an overview of how the ROXANNE platform supports analysts and investigators in key phases of their work.

Social Network Analysis for Criminology in ROXANNE

— by Garg, Shivam — last modified Nov 05, 2021 02:56 PM

Social network analysis provides an essential set of analytic tools to study people’s behavior: Social influence analysis, community detection, link prediction, and cross-network analysis are examples of such tools. Law enforcement agencies can benefit from them in criminology. In ROXANNE, we focus on several real crime cases and build a specialized tool for criminology that includes the state-of-the-art methods for each of these options.

ROXANNE Training Framework: Delivering the training material to end-users through remote (or physical) training sessions

— by Garg, Shivam — last modified Nov 05, 2021 02:57 PM

Learn facts about learning and our training methodology for effective use of the ROXANNE platform.

Ηow organized crime and cybercrime altered during the COVID-19 period

— by Garg, Shivam — last modified Nov 04, 2021 02:36 PM

The COVID-19 pandemic is expected to have a multifaceted impact on serious organized crime, particularly cybercrime. Lifestyle changes that arose in response to the restrictions during the epidemic, such as extended teleworking, widespread internet purchasing, and other behaviors, are unlikely to fade once the lockdown is lifted. As a result, cybercriminals will continue to seek ways to exploit these behaviors, either by modifying existing attacks or developing new ones.

Combining computer vision techniques with speech technologies for the fight against crime

— by Garg, Shivam — last modified Nov 05, 2021 02:58 PM

ROXANNE combines multiple analytics on various modalities (audio, text, image, metadata) to support LEAs in their investigations. In this post, we review how analysts and investigators can benefit from the significant progress achieved in computer vision in recent years, especially when combining it with speech or network analysis.

Dissemination & exploitation efforts in project ROXANNE

— by Garg, Shivam — last modified May 31, 2021 10:43 AM

ROXANNE project has completed about half of its journey until now. We are happy to share that with the support, feedback, and comments of our readers, stakeholders, end-users, and partners, we have achieved to create a preliminary version of this unique platform. We are certain that this platform will be used ethically for the betterment of society and we are ensuring checks for the same at various steps as well. In this blog, we will discuss some of the recent project updates and explore our efforts and objectives with respect to exploitation.

NLP technologies against online crime

— by Garg, Shivam — last modified Apr 27, 2021 09:41 AM

Big Data in Artificial Intelligence (AI) and Machine Learning (ML) technologies offer new opportunities. The principle is that an extensive amount of data is required for the best result of the scenario of fighting crime ML models. In the ROXANNE project, the technical development targets to significantly enhance the criminal network analysis based on text, speech, language, and video technologies.

ROXANNE Stakeholder Board

— by Garg, Shivam — last modified Apr 01, 2021 11:00 AM

As an EU-funded collaborative research and innovative project, ROXANNE has been accompanied since its launch by a Stakeholder Board (SB) for independent and methodological advice, counseling the consortium on the practical implementation of its goals, as well as on encountered issues regarding its progress, direction, and outcomes. This external project advisory group is formed of 16 interdisciplinary experts coming from diverse backgrounds, including law enforcement, academia, EU entities, international organisations, fellow research projects, and industry.

Multimodal networks integration improves the criminal entity network construction with the help of deep neural networks

— by Garg, Shivam — last modified Nov 05, 2021 02:58 PM

The network integration fuses information from ROXANNE platform component technologies for network analysis. In previous workloads, crime-related networks from different data sources are constructed. We propose a multilayer and cross-network structural analysis to provide methods for integrating information from several investigation cases and linking the data records across these.

Making Natural Language Processing work for Little Training Data

— by Garg, Shivam — last modified Nov 05, 2021 02:59 PM

In the ROXANNE project we are solving real-world problems which often do not have a large labelled training set. To overcome such data scarcity problems, we use the combination of distant supervision and noise-robust learning algorithms. Distant supervision serves as a way of obtaining a large amount of labelled data in an automatic manner. Such data is however often noisy so that feeding them directly to DNN models will hinder the learning process. Nonetheless, noise-robust learning algorithms will help the models to focus on the correct instances in the training set so that the models will not be affected much by the noise.

The Benefits of Using Automatic Speaker Recognition in LEAs’ Investigative Processes

— by Garg, Shivam — last modified Nov 05, 2021 03:00 PM

Traditional manually performed forensic analyses are language-dependent and time-consuming, while automatic speaker recognition systems are fast and language-independent (nevertheless, the role of a forensic expert is still irreplaceable for the presentation of the results). There is also other voice biometric information that can be automatically extracted from a person’s voice, such as the gender of a speaker. The bottom line is that current automatic speaker recognition systems, such as the one currently built by the ROXANNE consortium, can greatly improve the analytical work efficiency, enabling law enforcement agencies of any size to investigate faster.

Probabilistic modelling for speaker recognition in criminal networks

— by Garg, Shivam — last modified Nov 05, 2021 03:00 PM

Speaker recognition is a technology that uses computer algorithms to analyze speech patterns and determine the identity of the speaker in a recording. Speaker recognition is an integral part of the ROXANNE platform because the identities of speakers in recordings from criminal investigations are usually not known. However, the speaker recognition scenario in criminal investigations differs from the most other speaker recognition scenarios because in addition to the audio of the recordings, we also have access to information about how the recordings are related. In particular, information about who has talked to whom is important. This information forms a network structure between the recordings. Naturally, it is important for the investigation to accurately uncover the network structure. Interestingly, using prior knowledge about the expected network structure in criminal networks may also improve the accuracy of speaker recognition.

Technology to enhance forensic speaker analysis

— by Garg, Shivam — last modified Nov 05, 2021 03:01 PM

Law enforcement agencies investigate criminal networks to find participants, understand their role in the network and, eventually, collect evidence for prosecution. Opportunities to identify criminals often lie in the traces that their communication leaves behind. A cell phone leaves many different traces, but the availability and usability of these modalities may vary. Therefore, it is worthwhile to invest in multiple modalities, increasing the chance of a useful result.

Ethics Oversight in ROXANNE

— by Garg, Shivam — last modified Nov 05, 2021 03:01 PM

Ethics oversight in ROXANNE exists across four levels: the European Commission Ethics Checks; the External Ethics Board; the Internal Ethics Board; ethics-focussed work within the project. Overall, the ROXANNE project is in a positive position in terms of meeting the strict ethical requirements from the Commission. So far, the project is also fulfilling the recommendations generated within the project to ensure compliance with ethical, societal, and legal standards. The strict oversight that the ROXANNE project is subject to ensures that none of these standards are violated in the work of the project. The ROXANNE consortium is committed to ensuring that the ROXANNE project is carried out in a responsible way, and that the technologies developed are used in an ethical and lawful manner.

Link prediction algorithms to enhance criminal network analysis

— by Garg, Shivam — last modified Nov 05, 2021 03:02 PM

Roxanne project will include link prediction algorithms to enhance criminal network analysis by law enforcement agencies. A recent study shows that link prediction can predict links among criminals, although the effectiveness and accuracy of the prediction may be affected by the amount of unknown information and the specific type of relational data under analysis

Ethically developed technologies for safer societies – The ROXANNE project case study

— by Garg, Shivam — last modified Mar 01, 2022 11:12 AM

In order to implement an approach that incorporates ethical and legal concerns in ROXANNE, there are two research focuses that specifically deal with ethical and legal concerns which are led by Trilateral. The first area of focus ensures that the research carried out during the ROXANNE project abides by the standards of research ethics and that data protection legislation is applied to training algorithmic models used in the platform. A second research focus considers the issues which ROXANNE could raise in terms of ethics, societal values, fundamental rights, and applicable law if it is used in real investigations.

Overview of LTEC Voice Databases & ASR System Training

— by Garg, Shivam — last modified Nov 05, 2021 03:03 PM

The main problem in the application of ASR systems in forensics is the accuracy and reliability of the results of such system. The accuracy of identification methods depends on a number of factors that cannot always be assessed. Since it is very difficult to assess the impact of all the factors encountered in forensic speaker examinations, the performance of such systems can best be determined using voice databases developed on the basis of audio recordings submitted for examinations. Despite the variety of created voice databases that attempt to record voices under a variety of conditions, forensic investigations still encounter factors, whose impact on an automated speaker recognition system, is often unknown. ROXANNE BLOG briefly presents characteristics of voice databases - BALSAS_LTv1.1 and BALSAS-200LT of the Forensic Centre of Lithuania (LTEC), also describes ASR VOICE training performance using BALSAS_LTv1.1. In the ROXANNE project, LTEC will use the described voice databases (BALSAS_LT200 and BALSAS_LTv1.1) to assess the recognition accuracy of the existing ASR systems VOICE and BATVOX, as well as the ones developed internally.

Forensic Automatic Speaker Recognition (FASR) : Problems and prospects

— by Garg, Shivam — last modified Feb 05, 2021 04:02 PM

At present speaker identification by voice in forensics, criminalistics and other applications receive a great deal of attention and huge financial resources. In many cases, this is due to the increasing use of sound recording devices, and in particular, the widespread use of mobile technologies in criminal activities and their recordings, and accordingly, the use of various latest trending technologies in the fight against organised crime and international terrorism. In the ROXANNE project, speaker recognition is one of the main and most important elements of the system. However, direct application of automatic speaker recognition (ASR) systems in forensics raises a number of issues. In general, ASR methods work well only under controlled conditions, sufficiently good signal quality and relatively long duration. The main problem in the application of ASR systems in forensics is the accuracy and reliability of the results of such system. This blog briefly describes problems arising using this systems in forensic speaker identification, and prospects of effective using ASR system with traditional auditory-instrumental methods.

Automatic Speech Recognition: Setting, Benefits and Limitations

— by Garg, Shivam — last modified Nov 05, 2021 03:03 PM

Audio is an important modality for criminal investigations. This is true for different kinds of audio and in particular for spoken content. Be it a part of files, streams or posts which are accessible on the web and social media, a recording collected from mobile devices for investigations and interviews, or a conversation recorded via intercepted telephone calls - speech can be encountered in many environments. Audio data by itself, however, presents unstructured information. Without having access to the spoken content, no further analysis and processing is possible. Enter Automatic Speech Recognition (ASR).

Legally compliant state-of-the-art capabilities to fight and prevent transnational crimes

— by Garg, Shivam — last modified Nov 05, 2021 03:04 PM

With the increased reliance on new and rapidly evolving technologies to commit and facilitate crime, law enforcement agencies (LEA) must keep pace with the constant technological developments to stay ahead of criminals and maintain a safe environment. The ROXANNE advanced technical platform aims to enhance the quality of investigative work in tracking and identifying criminals by developing an innovative solution that combines speech, text and video data analysis with the analysis of organized crime networks. To be effective and lawfully implemented, such innovative police technologies must be part of an appropriate legal framework based on the principle of law, fundamental human rights and other applicable legislation ( Often innovative fields, such as data analysis for law enforcement purposes and the use of novel analytical methods, are subject to legal uncertainty due to outdated legal texts or insufficient guidance from legislators, be it domestic or regional)

Forensics Visualizations as a catalyst for fighting organized crime

— by Garg, Shivam — last modified Nov 05, 2021 03:04 PM

Criminals often employ sophisticated technologies in planning and executing their illegal activities, as well as concealing the revenues stemming from them. Suppressing organized crime remains an important but challenging task for Law Enforcement Agencies (LEAs) for several reasons. In this context, visualisation plays a key role in augmenting the human analysis and providing context to the analyst.

ROXANNE platform & network analysis

— by Garg, Shivam — last modified Nov 05, 2021 03:05 PM

Maximizing the speed of investigations is one of the key factors in the presence of dynamically changing and technologically savvy organized criminals. A number of automated network analysis methods were developed in the past that can be borrowed to help investigators to identify key players, paths and relations within the resulting networks in order to analyse the development of the network, find effective strategies toward its destabilisation or disruption and overall to increase investigation effectiveness.

Combination of Speech and Text Technologies with Criminal Network Analysis: Steps Toward First Field-Test Event of ROXANNE Project

— by Garg, Shivam — last modified Nov 05, 2021 03:06 PM

For law enforcement agencies (LEAs) to keep up in this new environment, they must change their approach to criminal investigations as relying mainly on physical evidence and witness statements is no longer sufficient in many cases. Training in research-based investigative procedures and access to related tools and resources can help law enforcement officers carry out successful investigations. It can help LEAs solve cases more swiftly and even prevent crimes in some cases.