Skip to main content

Data-In Integration for Large-Scale Imports

Learn how to enable and use the Data-In Integration for large-scale first party data uploads.

J
Written by Juan Sebastian Franco
Updated over 2 months ago

The Data-In Integration feature allows users to upload large audience files to Loop through an S3-based process. This is the preferred method for:

  • Repetitive large file uploads that exceed UI limitations.

  • Files larger than 1GB, which cannot be uploaded manually.

File Format and Structure

To ensure proper ingestion, all files must follow a standard structure.

File Requirements

  • Format: Tab-Separated Values (TSV) (.tsv extension required).

  • Compression: Files may be compressed in GZIP format.

  • Headers: The first row must contain column headers matching the supported fields names.

  • Encoding: UTF-8 encoding is required.

  • No Spaces in Filenames.

  • TAB (\t) as Field Separator, comma (,) for segment lists.

  • Hashed Data: Plain-text values for the device ID field are automatically hashed to SHA-256 during processing.

  • New Line at the end of each row: Carriage Return \r is NOT supported).

Supported Fields

Field Name

Description

Variable Type

Possible Values

device_type

Type of user device. Required

String

android, ios, web, external_id, email, phone, nid, rely_encrypt

device_id

Unique identifier of the user’s device Required. This can also be an email, phone number, or national ID, as explained below.

String

UUID format (00461ac5-f632-4e2e-8ae8-e7429eba81a4) or phone, email or national ID as explained below.

country

Country identifier

String

AR, CL, CO, BR, PE, EC, UY, MX

segments

Comma-separated list of segment IDs of the segments where this data should be populated in. Required

String

Comma separated Numeric segment IDs (e.g., "101101,101456,101789")

country_code

Optional country code

String

Country abbreviation (e.g., US, BR)

partner_segment_id

Internal partner segment ID for mapping

String

Custom value

remove_segments

List of segments to remove from a device

String

Numeric segment IDs (e.g., 123,456,789)

remove_partner_segment_id

Remove partner-specific segment IDs

String

Custom value

Device ID and Type Considerations

Device Type

Description

web

Device ID is a Retargetly web cookie.

android

Device ID is an Android Advertising ID (ADID).

ios

Device ID is an Apple ID for Advertisers (IDFA).

external_id

Partner-assigned external identifier for a user.

email

Email-based identifier. Must be in lowercase or hashed (SHA-256, SHA-512).

phone

Numeric phone number in (country code)(area code without 0)(number) format.

nid

National ID (e.g., DNI, CI). Only numeric values allowed.

rely_encrypt

Encrypted device ID requiring prior key generation.

Correct Format Example

device_type device_id country_code segments
email [email protected] MX 111111,22222,33333
email fc2ae4a1fb374548ea80556dc51ab3471a311231d8bffaa1dece31371bcceb62 AR 123456,123410,232322
android 1b671a64-40d5-491e-99b0-da01ff1f3341 AR 123123

Common Mistakes When Creating a TSV File for Data-In Integration

Proper formatting is crucial for a successful Data-In upload in Loop. Below are common errors that can cause processing failures and how to avoid them.

1. Missing Header Row

The first row of the file must contain the correct column headers. If missing, the system won’t know how to interpret the data.

Incorrect (Missing Headers)

email [email protected] 111111,22222,33333 android 1b671a64-40d5-491e-99b0-da01ff1f3341 123123

Correct (Headers Included)

device_type device_id segments email [email protected] 111111,22222,33333 android 1b671a64-40d5-491e-99b0-da01ff1f3341 123123

2. Invalid Device Type

The device_type column must contain a valid device type (e.g., android, ios, email). Any unrecognized value will cause errors.

Incorrect (Invalid Device Type)

device_type device_id segments android 1b671a64-40d5-491e-99b0-da01ff1f3341 123123 e-mail [email protected] 111111,22222,33333

Correct (Standard Device Types Only)

device_type device_id segments android 1b671a64-40d5-491e-99b0-da01ff1f3341 123123 email [email protected] 111111,22222,33333

3. Incorrect Field Separator (Must Be TAB)

TSV (Tab-Separated Values) files must use tabs (\t) as the column separator. If semicolons (;) or commas (,) are used instead, the data won’t be processed correctly.

Incorrect (Using Semicolons Instead of Tabs)

device_type;device_id;country_code;segments email;[email protected];MX;111111,22222,33333 android;1b671a64-40d5-491e-99b0-da01ff1f3341;AR;123123

Correct (Using Tabs as Separators)

device_type device_id country_code segments email [email protected] MX 111111,22222,33333 android 1b671a64-40d5-491e-99b0-da01ff1f3341 AR 123123

4. Missing Required Fields (Device Type, Device ID, or Segments)

Every row must include:

  • A valid device type (email, android, ios, etc.).

  • A device ID or hashed PII (email, phone, or national ID).

  • At least one segment ID (comma-separated if multiple).

Incorrect (Missing device_type and segments)

device_id [email protected] fc2ae4a1fb374548ea80556dc51ab3471a311231d8bffaa1dece31371bcceb62 1b671a64-40d5-491e-99b0-da01ff1f3341

Correct (All Required Fields Included)

device_type device_id segments email [email protected] 111111,22222,33333 android 1b671a64-40d5-491e-99b0-da01ff1f3341 123123

Best Practices

✅ Follow the file formatting guidelines to avoid ingestion errors.
✅ Compress large files in GZIP format for faster processing.

Did this answer your question?