kaggle-cli-submission

name	kaggle-cli-submission
description	Run a full end-to-end Kaggle competition workflow with the Kaggle CLI (kaggle): read competition pages/rules, download data, develop a portable .py script kernel, push with competition_sources (twice), poll kernels status, verify submission.csv output, submit a kernel version to a code competition, and poll publicScore. Use when submitting to Kaggle, automating kaggle kernels push/submit, working on code competitions, or when the user mentions Kaggle CLI submission flow.
compatibility	Requires the kaggle CLI installed and authenticated via KAGGLE_API_TOKEN or ~/.kaggle/access_token; network access to kaggle.com
metadata	{"author":"ioai-isc"}

Kaggle CLI — Full End-to-End Competition Flow (Agent Playbook)

This document is a complete, self-contained playbook for an autonomous agent to participate in a Kaggle competition using the Kaggle CLI (kaggle). It assumes the CLI is already installed, and covers reading the competition, downloading data, developing a solution, running it on Kaggle, submitting, and reading the score — with the exact commands, the machine-readable output flags, the pitfalls, and how to recover from each failure.

You submit a plain Python .py file — NOT a Jupyter .ipynb. Kaggle calls the thing you submit a "notebook" (or "kernel"), but a kernel can be a script: an ordinary .py file with normal top-to-bottom Python code. Set "kernel_type": "script" in the metadata and point code_file at your .py. There are no notebook cells, no JSON cell structure, and no .ipynb involved. Develop and iterate on a regular .py file exactly as you normally would, then push it. (.ipynb notebooks are also supported via "kernel_type": "notebook", but you do not need them and this guide uses scripts throughout.)

---

0. The whole flow in one glance

read rules/description   ->  kaggle competitions pages <C> --page-name <p> --content
list data files          ->  kaggle competitions files <C>
download data            ->  kaggle competitions download <C> -p data/    (then unzip)
[develop + test locally using downloaded data]
push .py script + data   ->  kaggle kernels push -p kernel/     (PUSH TWICE)
wait for run             ->  kaggle kernels status <owner>/<slug>   (poll until COMPLETE)
inspect output/logs      ->  kaggle kernels output <owner>/<slug> -p out/
submit the script        ->  kaggle competitions submit <C> -k <owner>/<slug> -v <ver> -f submission.csv -m "msg"
read score               ->  kaggle competitions submissions <C> --format json   (poll until score)

<C> = competition slug (e.g. titanic). <owner>/<slug> = your kernel id.

---

1. Authentication

Get a token at kaggle.com → Settings → Create New Token (starts with KGAT_). Then provide it in one of two ways:

export KAGGLE_API_TOKEN="KGAT_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"   # simplest for agents
# OR
mkdir -p ~/.kaggle && printf 'KGAT_xxxx' > ~/.kaggle/access_token && chmod 600 ~/.kaggle/access_token

The env var is the cleanest for automation. Verify auth with any authenticated call:

kaggle kernels list -m        # lists YOUR kernels; a normal response == authed

Note: kaggle auth print-access-token only works for the interactive OAuth login and will say "you must log in" even when the token env var is working correctly. Ignore it; test auth with a real API call like the one above instead.

Scripting hygiene

• Add --format json to get machine-readable output where supported (competitions files, competitions submissions, competitions list, etc.).
• Add -q / --quiet to suppress progress bars in non-interactive runs.

---

2. Understand the competition (read before coding)

2.1 Is it a notebook-only ("code") competition?

This determines the entire submission mechanism. A code competition requires you to submit a kernel (which, despite Kaggle calling it a "notebook," can be a plain .py script — see §4.1), not a raw CSV. Kaggle re-runs your code to produce and score the output. If the rules say "submissions must be generated by a Kaggle Notebook," treat it as a code competition and follow this whole guide. (A regular competition instead lets you upload a CSV directly — see the note in §6.)

2.2 List and read the content pages

kaggle competitions pages <C>                                   # list page names
kaggle competitions pages <C> --page-name data-description --content
kaggle competitions pages <C> --page-name Evaluation --content
kaggle competitions pages <C> --page-name rules --content

Read these carefully and extract:

• What to predict and the exact submission file format (columns, id format).
• The output filename expected (commonly submission.csv).
• The metric.
• Constraints: internet on/off, allowed data/models, step/row limits, and the daily submission cap.

2.3 Inspect the data files

kaggle competitions files <C>                    # names + sizes
kaggle competitions files <C> --format json      # machine-readable

---

3. Download data and develop locally

Download the data and test your solution locally before spending a Kaggle run.

mkdir -p data
kaggle competitions download <C> -p data/            # downloads a .zip
cd data && unzip -o "<C>.zip" && cd ..               # extract
# single file instead of all:
kaggle competitions download <C> -f <file> -p data/

Write your solution so it works both locally and on Kaggle. The key portability rule: do not hard-code the Kaggle input path. On Kaggle the competition data appears under /kaggle/input/..., and the exact subfolder varies (it can be /kaggle/input/<C>/ OR /kaggle/input/competitions/<C>/). Locate files by name:

from pathlib import Path

def find_input(name, local="data"):
    # On Kaggle, search the mounted inputs; locally, use ./data
    for base in ("/kaggle/input", local):
        matches = list(Path(base).rglob(name))
        if matches:
            return matches[0]
    raise FileNotFoundError(name)

test_path = find_input("test_scenarios.pkl")

Write the submission to /kaggle/working/submission.csv on Kaggle (and a local path when testing). Add cheap assertions so a bad run fails loudly instead of emitting a silently-wrong file (row count, no duplicate ids, required columns present).

Run it locally against the downloaded data to confirm it produces a well-formed file before pushing.

---

4. Push the notebook (with competition data attached)

4.1 Create the kernel folder

Put your code file and a kernel-metadata.json in one folder. The code file is a plain Python .py script — the same file you developed and tested locally. No .ipynb, no cells.

kernel/
├── script.py                 # your ordinary .py solution (kernel_type "script")
└── kernel-metadata.json

Generate a metadata template with kaggle kernels init -p kernel/, then edit it. A working metadata file for a code-competition script:

{
  "id": "YOUR_USERNAME/your-kernel-slug",
  "title": "Your Kernel Title",
  "code_file": "script.py",
  "language": "python",
  "kernel_type": "script",
  "is_private": "true",
  "enable_gpu": "false",
  "enable_tpu": "false",
  "enable_internet": "false",
  "machine_shape": "",
  "dataset_sources": [],
  "competition_sources": ["<C>"],
  "kernel_sources": [],
  "model_sources" 

Field notes (the ones that matter):

• id: username/kernel-slug. The slug must be lowercase-with-dashes and match the title lowercased. Kaggle creates it on first push, versions it on later pushes.
• code_file: path (relative to the metadata file) to your source.
• kernel_type: use script — this makes code_file a plain .py file that runs top-to-bottom, with no cells and no .ipynb. (The alternative notebook expects a .ipynb; you don't need it.) Cannot be changed after creation, so pick script from the start.
• competition_sources: ["<C>"] — this is what mounts the competition data under /kaggle/input. Use the competition slug. For datasets use dataset_sources: ["owner/dataset"].
• enable_internet: set "false" — most code competitions require offline runs, and the data is mounted locally anyway.
• is_private: keep "true". Making a notebook public requires phone verification and is unnecessary for submitting.
• machine_shape: leave "" for CPU. For a GPU, use e.g. "NvidiaTeslaT4" or "NvidiaTeslaP100"; for TPU "Tpu1VmV38". (You can also pass --accelerator on push.)

4.2 Push — and push TWICE

kaggle kernels push -p kernel/
kaggle kernels push -p kernel/        # push again

Why twice: when a competition source is newly attached to a kernel, the first run frequently starts before the data is mounted, so it fails with FileNotFoundError on the data. Pushing the identical folder a second time triggers a fresh run that has the data mounted. Always push twice on first creation. (On later updates to an existing, already-attached kernel, a single push is usually enough — but a second push is a cheap safety net.)

Each push prints the new version number: Kernel version N successfully pushed. Record N — you need it to submit.

---

5. Wait for the run, then verify it produced the submission

5.1 Poll status until terminal

kaggle kernels status YOUR_USERNAME/your-kernel-slug

Returns one of: KernelWorkerStatus.QUEUED, .RUNNING, .COMPLETE, .ERROR. Poll every ~15–30s until COMPLETE or ERROR. (Small notebooks finish in well under a minute, but allow for queueing/imaging.)

Agent loop sketch:

for i in $(seq 1 40); do
  s=$(kaggle kernels status YOUR_USERNAME/your-kernel-slug 2>&1)
  echo "$s"
  case "$s" in
    *COMPLETE*) break ;;
    *ERROR*)    echo "RUN FAILED"; break ;;
  esac
  sleep 20
done

5.2 Download output + logs and verify

mkdir -p out
kaggle kernels output YOUR_USERNAME/your-kernel-slug -p out/
ls out/                       # expect submission.csv (+ a .log)

• If submission.csv is present and the log shows your success print → good.
• If status was ERROR, open the .log in out/ — it contains the full stderr traceback. The most common failure is the data-not-mounted FileNotFoundError (fix: push again per §4.2). Fix the code or re-push, then repeat §5.

What submission.csv actually is: it is the file your notebook writes on Kaggle during the run (/kaggle/working/submission.csv). It is NOT uploaded from your machine. In the submit step below, -f submission.csv merely selects which output file of the run to score — it transfers no local data. Your score comes from the file your code generated during Kaggle's controlled re-run (internet off, data mounted), which is what makes code competitions tamper-resistant.

---

6. Submit the notebook to the competition

kaggle competitions submit <C> \
  -k YOUR_USERNAME/your-kernel-slug \
  -v <VERSION> \
  -f submission.csv \
  -m "short description of this submission"

• -k: the kernel id (owner/slug).
• -v: the version number that ran to COMPLETE (from the push output). Must be a version whose run succeeded and produced the output file.
• -f: the output filename to score (usually submission.csv).
• -m: submission message.
• Do not use --sandbox (that's for competition hosts/admins only).

A successful submit prints a confirmation (or nothing). Errors are printed to stderr — capture and inspect them.

Regular (non-code) competition: upload a local file instead: kaggle competitions submit <C> -f path/to/local_submission.csv -m "msg". There, -f is a real upload from your disk. Code competitions do NOT use this; they use -k/-v/-f as above.

---

7. Read the score (poll until scored)

kaggle competitions submissions <C> --format json

Each submission has status and publicScore/privateScore. Right after submitting, the score may be empty ("still scoring"); poll every ~30s until status is COMPLETE and publicScore is populated.

Agent loop sketch:

for i in $(seq 1 20); do
  j=$(kaggle competitions submissions <C> --format json 2>/dev/null)
  echo "$j" | python -c "import sys,json; d=json.load(sys.stdin)[0]; print(d['status'], d['publicScore'])"
  echo "$j" | grep -q '"publicScore": "[0-9]' && break
  sleep 30
done

You can also see standings with kaggle competitions leaderboard <C> --show.

Respect the daily submission limit stated in the rules (e.g. 5/day). Track how many submissions you've made so an agent doesn't burn the quota.

---

8. Command reference

Goal	Kaggle CLI command
Read description / rules / evaluation	kaggle competitions pages <C> --page-name <name> --content
List data files	kaggle competitions files <C> (--format json)
Download all data	kaggle competitions download <C> -p data/
Download one file	kaggle competitions download <C> -f <file> -p data/
Leaderboard	kaggle competitions leaderboard <C> --show
Create/push notebook with data attached	kaggle kernels push -p kernel/ (metadata has competition_sources)
Run status	kaggle kernels status <owner>/<slug>
Logs + output files	kaggle kernels output <owner>/<slug> -p out/
Submit notebook to code competition	kaggle competitions submit <C> -k <owner>/<slug> -v <ver> -f submission.csv -m "msg"
Poll score	kaggle competitions submissions <C> --format json
Find your notebooks	kaggle kernels list -m
Metadata template	kaggle kernels init -p kernel/

---

9. Pitfalls and recovery (agent troubleshooting)

Symptom	Cause	Fix
Notebook errors FileNotFoundError on the data file	Data not mounted (first run after attaching source), or hard-coded input path	Push again (§4.2); use rglob to find files (§3)
submission.csv missing from kernels output	Run errored, or code didn't write to /kaggle/working/	Read the .log; fix code; ensure output path is /kaggle/working/submission.csv
Submit rejected: "Notebook must include this competition as a data source / output file not found"	Submitted a version with no competition_sources or a failed run	Ensure competition_sources: ["<C>"] in metadata; submit a COMPLETE version that produced the file
Submit rejected: version/permission error	Passed a wrong version number	Use the integer -v version printed by kernels push
kernels status / output "not found"	Kernel hasn't committed a completed version yet	Wait; a version must run before it's queryable
"Phone verification is required to make a notebook public"	Tried to make the kernel public	Keep is_private: "true"; private notebooks submit fine
Auth errors on every call	Token not picked up	export KAGGLE_API_TOKEN=KGAT_... or write ~/.kaggle/access_token; retest with kaggle kernels list -m
Data access denied	Haven't accepted the competition rules	Accept rules on the competition website once, then retry
competitions list -s <slug> shows nothing	Private/community competitions aren't in public search	Reference the competition directly by slug in every command (works fine)
Daily submission cap hit	Exceeded the rules' per-day limit	Wait until the cap resets; track submissions to avoid this

---

10. Copy-paste end-to-end template (fill in the CAPS)

set -euo pipefail
export KAGGLE_API_TOKEN="KGAT_XXXX"
C="COMPETITION_SLUG"
USER="YOUR_USERNAME"
SLUG="your-kernel-slug"

# 1) Understand the task
kaggle competitions pages "$C" --page-name data-description --content
kaggle competitions pages "$C" --page-name Evaluation --content
kaggle competitions files "$C"

# 2) Get data and develop/test locally
mkdir -p data && kaggle competitions download "$C" -p data/ && (cd data && unzip -o "$C.zip")
#   ... write kernel/script.py so it uses find_input() and writes /kaggle/working/submission.csv
#   ... run it locally against ./data to validate the output format ...

# 3) Kernel metadata (kernel/kernel-metadata.json) MUST include:
#     "id": "'"$USER"'/'"$SLUG"'", "code_file": "script.py",
#     "kernel_type": "script", "language": "python",
#     "is_private": "true", "enable_internet": "false",
#     "competition_sources": ["'"$C"'"]

# 4) Push (twice) so the data mounts; note the version number it prints
kaggle kernels push -p kernel/
kaggle kernels push -p kernel/

# 5) Wait for COMPLETE
for i in $(seq 1 40); do
  s=$(kaggle kernels status "$USER/$SLUG" 2>&1); echo "$s"
  case "$s" in *COMPLETE*) ;; *ERROR*)  FAILED; kaggle kernels output  -p out/;  1;; 
   20



 -p out && kaggle kernels output  -p out/ &&  out/


VER=
kaggle competitions submit  -k  -v  -f submission.csv -m 


 i  $( 1 20); 
  j=$(kaggle competitions submissions  --format json 2>/dev/null)
    | python -c 
    | grep -q  && 
   30

Track the exact version number from each kaggle kernels push output (it prints Kernel version N successfully pushed.) and pass that N to -v. Don't guess it.

---

11. Mental model (one paragraph)

A code competition never scores a file you upload; it scores a file your code produces when Kaggle re-runs it with the internet off and the competition data mounted at /kaggle/input. The code you submit is a plain .py script (Kaggle labels it a "notebook," but it's just a script — no cells, no .ipynb). So the whole job is: (1) read the rules/format, (2) download the data to develop and validate locally as an ordinary Python file, (3) push that .py as a kernel whose kernel-metadata.json lists the competition under competition_sources so the data mounts, (4) let it run to COMPLETE and confirm it wrote submission.csv, (5) submit that kernel version, selecting submission.csv as the output to score, and (6) poll the submissions list until a publicScore appears.