Upload CSV files to Conduit via API
Using our No-code editor, you can create a virtual table that will be populated with data from your CSV files.
Steps to Send Data:
- Start by creating a new API integration.
- In this integration, define the virtual table and copy the URL provided.
- Depending on your scenario, add one or more parameters to this URL.
- Utilize this URL in your script or application to send data.
Our API supports various scenarios for working with this virtual table:
- Inserting data for the current day
- Inserting rows with data from a specific day in the past
- Inserting rows with data from multiple days in the past
- Updating rows
Please note that individual rows cannot be deleted. You can only erase all rows from a given day.
Create a new integration
Before sending data, complete the necessary API configuration.
Find the “Push API” integration https://app.getconduit.app/credentials/?q=push+api
Click the Connect button
Enter a name for the integration
Set up columns for your virtual table
- Add a new column
- Name your column.
- Select the appropriate data type for dimensions.
- Name another column.
- Select “Metric” for numeric columns.
- Specify a particular data type for numeric columns.
- Copy the URL and incorporate it into your scripts.
Click the “Save” button.
The integration is ready. You can create your script.
Basic Usage
Uploading Files
To upload a file, make a POST request to the provided URL. You can specify a date by appending ?date=YYYY-MM-DD
to the URL; otherwise, today's date is used based on your profile's time zone settings.
Example using curl:
curl -X POST -F "file=@stats_2024_01_01.csv" "https://api.getconduit.app/push/{uniqtoken}/receive/?date=2024-01-01"
Replace {uniqtoken} with a real token.
You'll receive a response with a file ID, for instance:
{"event_id": 1}
(Optional)Checking File Status
File processing is asynchronous. To check a file's status, use its file ID in a GET request.
Example using curl:
curl "https://api.getconduit.app/push/{uniqtoken}/status/1/"
Responses indicate processing success or failure and provide details accordingly.
Examples of API Responses:
- File not yet processed:
{"processed": true, "failed": false, "details": null}
- Successful processing:
{"processed": true, "failed": false, "details": null}
- Unsuccessful processing:
{"processed": true, "failed": true, "details": "[Campaign] column not found"}
CSV File Requirements:
- The first row must include header names corresponding to your specific API configuration.
- Columns should be separated by commas.
- Use double quotes to escape characters.
- Do not include thousand separators in numbers; use a period for decimals.
- Format dates as YYYY-MM-DD and datetime values as YYYY-MM-DDTHH:MM:SS.
- Include time zone in numeric format (+0300) for datetime; UTC is default if unspecified.
Limitations
- Maximum file size: 10 MB.
- Rate limit: No more than one API call per second.
After Processing
Once processed, data becomes accessible within Conduit's various modules, including Copilot, Imports, Workflows, and Dashboard.
Upon uploading data, it is integrated into the virtual table. All separate CSV file uploads will be consolidated into a single continuous table.
You can update previously uploaded data chunks by specifying a specific DAY. See the ?date=YYYY-MM-DD
parameter.
Uploading a CSV file with multiple days
By default, the server updates rows for a single day, which is either today or the day specified in the date parameter.
If your CSV file contains data for multiple days, you can specify which column contains the date to be used as the key for selecting the appropriate day.
To do this, go to the integration settings, check the "Use the specified column as a primary dimension for date/time" option, and specify the column name. This column must contain date values.
Updating rows
When you send a CSV file, the rows from the new file will completely overwrite the rows for the specified day, which can be inconvenient for many use cases.
To replace an existing row, add the mode=update
parameter to the URL. It is recommended to use this parameter along with one indicating a specific day; otherwise, you risk updating the row on an unintended day.
With the mode=update
parameter, our server operates as follows:
- The server searches for a unique identifier for each row:
- You can provide a unique ID in the table.
- Alternatively, the server will create a unique identifier by calculating the hash for all dimensions (but not metrics) of the row.
If your data already has a natural unique identifier, it is best to use it directly. To do this, specify the name of the column in the "Column with a unique row ID" field within the integration settings.
The server will search for a row with this ID. If it finds the row, it will update it with the new data. If it doesn't find the row, it will add the new line to the table.
More Options
Mode
Optional parameter mode can be set to [append | update | overwrite].
- overwrite: default,overwrites the day.
- append: Appends rows at the end of the day.
All data types support all modes
Default value:
CSV - mode=overwrite
FORM - mode=append
HTTP FORM Example
To send a row with 3 columns:
- col1 = abc
- col2 = 123
- col3 = 2024-01-01
curl -X POST "https://api.getconduit.app/push/{uniqtoken}/receive/?date=2024-01-01"
-d "col1=abc"
-d "col2=123"
-d "col3=2024-01-01"
JSON Submission Example
To send the same data using JSON:
curl -X POST "https://api.getconduit.app/push/{uniqtoken}/receive/?date=2024-01-01"
-H "Content-Type: application/json"
-d '{"col1": "abc", "col2": "123", "col3": "2024-01-01"}'
Content-Type
Supported HTTP Header Content-Type values:
- multipart/form-data
- application/x-www-form-urlencoded
To send CSV, you must specify the header Content-Type = multipart/form-data
For FORM, all three types of headers are supported. However, if JSON is being sent, the Content-Type header must be set to application/json.
Payload Type Specification
Optionally, the user can specify the parameter ?payload_type=[csv | form] to indicate the type of content being sent.
application/json
Limits
- File size limit: 10 MB
- HTTP BODY size limit for FORM: 10 KB
Rate Limits
- File requests: 1 request per 5 seconds.
- FORM requests: 5 requests per 5 seconds.