There are three key types of data you need to collect to process information from a Mothbox.
- Deployment
- Metadata
- Species List
Organization
Basic Organization
Just getting started? Wanna see them bugs?
1) Make yourself a folder called something like “Mothbox_Data” 2) Then dump all your folders you collect from the field of image data in there. (Like all the dated folders that have a bunch of images in them) 3) Choose this folder when you use Mothbot Process and Mothbot Classify
Advanced Organization
Once you start having tons of data from all over the world, you might want to get serious about your organization! Here’s how we do it! We organize our photographic data we get from “Deployments” of the Mothboxes like shown below:
.
└── Collection of Datasets/. # We call a collection of datasets a "Project"
├── DatasetA # These are the folders with the original, full size source images from a deployment/
│ ├── Deployment1/
│ │ └── devicename_YYYY-MM-DD #nightly folders (or however you might organize)/
│ │ ├── sourceimageA_YYYY-MM-DDTHHMMSS+00-00.jpg. #source image with device name and ANSI timestamp
│ │ └── sourceimageB_YYYY-MM-DDTHHMMSS+00-00.jpg. #second source image
│ ├── Deployment2
│ └── SiteA # Can optionally organize data by additional layer like a specific site with many deployments/
│ └── DeploymentA-1
├── DatasetB/
└── _processed #everything in this processed folder is created automatically by the software. Its directory structure is mirrored of the original datasets, but includes only lightweight processed data.
├── DatasetA/
│ ├── Deployment1/
│ │ └── devicename_YYYY-MM-DD/
│ │ ├── jsonfromsourceimageA.json # Json contains data connecting detection patch image to source image and other metadata
│ │ ├── patchfromsourceimageA_patch1.jpg
│ │ ├── patchfromsourceimageA_patch2.jpg
│ │ ├── patchfromsourceimageA_patch3.jpg
│ │ ├── jsonfromsourceimageB.json
│ │ ├── patchfromsourceimageB_patch1.jpg
│ │ └── patchfromsourceimageB_patch2.jpg
│ ├── Deployment2
│ └── SiteA/
│ └── DeploymentA-1
└── DatasetB
Optional Processing Files
In addition to the Deployment photo data, there are two other files you will need to completely process your data.
- Metadata CSV
- You can manually input this metadata in the software, but keeping a CSV of ALL your metadata can be useful for large scale deployments
- This ties the photos and IDs to metadata like location and date
- the column headings of the CSV should be
- Timestamp,device,firmware,sheet,schedule,dataset,project,site,latitude,longitude,height_above_ground,habitat,deployment_date,collect_date,data_storage_location,crew,notes,attractor,attractor_location,UTC,deployment_name
- Species List
- This improves the automatic Identification process by limiting the guess to only creatures that might be in your desired location and type of creature (e.g. Insecta or more broadly like Arthropoda)
- If you aren’t using a specific species list, our processing software will default to using a global list of all insects, so not big worry.
These two files don’t have to be organized in any special way, but we keep examples of these files in the AI folder of the github repo
Data Terminology Descriptions
It’s a weird new field of automated insect detection, and it can be useful to clarify our language. We are even working on a detailed description of these terms more broadly.
Below, we will discuss more particulars about how organize the Deployment, Metadata, and Species List.
Deployment
Each “deployment” is a data from device left out in the field somewhere and then recollected.
Deployment Name
The deployment has a unique name like this:
PROJECT_SITE_DEVICE_YYYY-MM-DD
The “Project” is a broad project that you are collecting this data for. You could name it something like “BatSurvey” or “MtTotumasDrySeason” (No spaces)
The “Site” is a human name for the very specific place you left the Mothbox, like “TreeNearLodge” (No Spaces)
The “Device” is a unique name that the Mothbox calls itself. These are names based off the internal serial number of the Raspberry Pi on the Mothbox meshed with a list we made of Spanish and English verbs, nouns, and adjectives. Like “FuerteFrog”
Then there is a date stamp that marks the first day a mothbox was left out in the field. like 2024-04-30. The format is YYYY-MM-DD.
Nightly Folders
A deployment usually has several nights. Each night is collected in its own folder.
The nightly folders are automatically created by the Mothbox and have a basic format:
YYYY-MM-DD
Mothbox “Nights”
A special note about Mothbox “nights.” Since most of our data collection happens at night, each night for these folders runs from 12:00 pm of the first day it is left out until 11:59am of the next day. In this way, images captured at, for instance, 3AM are considered part of the same night that started 10 hours earlier at 7 PM the preceding day.
Source Image
The “raw” photos we capture look like this. They are insects on a white background.
They have a name consisting of the device name and then a timestamp in ISO format
superDorada_2026-08-05T04-29-21-04-00.jpg
└devicename┘└YYYY-MM-DD┘└HH-MM-SS┘└±HH-MM┘ (UTC -4:00)
Data Labels
Source images can be associated with json labels that hold data about them.
- Bot created labels
These are files generated by automated means to detect where the insects are in the photo. Generally these files are made by the Mothbot_Detect.py script. Each sample photo might also have a similarly named file next to it, but the file type is “.json” and the file name ends with “botdetection.”
Samples
Each data “sample” consists of a set of grouped files.
superDorada_2026-08-05T04-29-21-04-00_botdetection.json
└devicename┘└YYYY-MM-DD┘└HH-MM-SS┘└±HH-MM┘└specifies it came from a robot┘
- Human created labels
There are files that have the same name as the Raw Image but end with “.json”. These are human-created “Ground-Truth” datasets. They don’t have “botdetection” on the ends their file names. They are made by programs like x-anylabeling.
superDorada_2026-08-05T04-29-21-04-00.json
└devicename┘└YYYY-MM-DD┘└HH-MM-SS┘└±HH-MM┘└same name as source file but ends in .json┘
Metadata
Equally as important as the photographic data you collect is the metadata about your deployments. We need to create a metadata file for each raw photo. This contains information about the sampling like:
- occurrenceID (file name with unique timestamp of the specific individual photo (“gradoVerdín_2024_07_25__21_12_05_HDR0_crop_0.jpg”)
- basisOfRecord (i.e. MACHINE_DETECTED)
- deployment ID
- eventDate (timestamp)
- GPS data
- raw_photo (location of the original “raw photo”)
- identifier (Who did the most up to date ID? i.e. “Mothbot” or “Hubert Szczygiel”
- cv_confidence (how confident the AI was in detecting this if machine detected)
- Taxonomic information: class order family genus species commonName scientificName
You should fill out a row on that form for each of your deployments.
We have printable forms that field technicians can take to their sites:
Alternatively, fill out
Remember to collect this metadata for EVERY SINGLE DEPLOYMENT or else it is not useful in the end!
All photos from a single deployment should be in a folder named with the convention: “PROJECT_SITE_MOTHBOXID_YYYY-MM-DD” (the COUNTRY_ prefix is optional)
Species List
The species list is used by the identification script to narrow down the possibilities of what it is trying to guess. Using GBIF’s species list generator, you can narrow down the possibilities by taxa or location. For example, you could download this list of only the insects that are in Panama.
If you want to go super broad, you could just try to get a list of all arthropods, or you could limit things to a specific family of moths. It’s up to you!
Start Processing
Go to the next steps in this section to start processing your data!