Using the Kaggle API to Download Data and Submit Predictions

Quick answer: download a dataset and submit a prediction with the Kaggle API

After you install the Kaggle CLI and place your API token, a single command fetches the competition files and another uploads your prediction file.

1. Install and configure the API

1. Install the package: pip install kaggle
2. Create a ~/.kaggle/ directory if it does not exist.
3. Copy your kaggle.json (username + key) into that directory and set permissions: chmod 600 ~/.kaggle/kaggle.json

2. Download the Titanic competition data

# Run from any directory; files will be placed in a folder named after the competition
kaggle competitions download -c titanic

The command creates titanic.zip; unzip it to access train.csv, test.csv, and the sample submission.

3. Prepare a submission file

For illustration, assume you have trained a model and saved predictions to submission.csv with exactly two columns: PassengerId and Survived, matching the sample format.

# Example of a minimal valid submission (replace with your own predictions)
echo "PassengerId,Survived" > submission.csv
# Append predictions; here we just copy the sample for demonstration
tail -n +2 sample_submission.csv >> submission.csv

4. Submit the file

kaggle competitions submit -c titanic -f submission.csv -m "Baseline submission from API guide"

The CLI returns a message with a submission ID; you can view the result on the competition’s Submissions page.

Verification steps

• Run kaggle --version to confirm the CLI is installed.
• Execute kaggle competitions list; an authenticated user sees a list of accessible competitions.
• After submission, open https://www.kaggle.com/competitions/titanic/submissions and check that your entry appears with the description you provided.

Limits and common mistakes

• Rate limits: The API allows roughly five requests per second; bursting beyond this may return HTTP 429 errors. Space out calls or add a short sleep in scripts.
• File format: Submission files must match the column names, order, and delimiter exactly as specified; any extra whitespace or missing header causes rejection.
• Path handling: Using relative paths can lead to “file not found” errors if the working directory changes; prefer absolute paths or verify pwd before referencing files.
• Credential safety: Never commit kaggle.json to version control; expose it and others can hijack your account. Add the .kaggle directory to .gitignore.
• Environment: The CLI requires internet access and a recent version of Python (≥3.6). Older systems may need a virtual environment or pip install --user kaggle.

By following these steps you can reliably automate data retrieval and submission for any Kaggle competition that offers the API.