VFrame by Adam Harvey: Machine learning for Human Rights researchers and Investigative Journalists
1040

VFRAME is a computer vision toolkit designed for human rights researchers and investigative journalists

VFrame

Project from Adam Harvey creates tools using Machine Learning for human rights researchers and investigative journalists – the video above demonstrates visual recognition of munition and shells:

Accelerating Human Rights Investigations with Computer Vision

People caught in conflict zones such as the Syrian conflict frequently post videos online that contain critical information for human rights investigations. But manually reviewing these videos is expensive, does not scale, and can cause vicarious trauma. As an increasing number of videos are posted, a new approach is needed.

VFRAME is a computer vision framework being developed to address these demands. It provides researchers and technologists with access to state-of-the-art tools to locate objects of interest, extract visual metadata, query related media, organize and annotate custom datasets, and provide filtering for traumatic content.

VFRAME is currently working directly with the Syrian Archive to establish the most effective and relevant tools for improving their workflow. Phase 1 of VFRAME will research and develop the following capabilities:

Visual Taxonomy
To understand how objects can be annotated for training datasets, they are first stuctured into a visual hierachy that defines and links signature visual characteristics
Custom Datasets
Using the visual taxonomy, datasets can be constructed using a new web-based annotation platform developed specifically for VFRAME
Object Detection
Using the custom datasets, state-of-the-art object detection algorithms are trained to locate and quantify objects of interest in large video datasets
Scene Metadata
Additional visual metadata such as scene attributes will provide researchers with more keywords to search and link videos
Graphic Filtering
Manually reviewing videos from conflict zones can cause vicarious trauma. VFRAME is creating graphic filtering capabilities to provide relief from reviewing traumatic content
Visual Metadata API
VFRAME converts videos into RESTful API endpoints that can be integrated into other workflows and custom search engines
DEMO

Related Posts

Recent Posts