modelscope-lfs-upload

name	modelscope-lfs-upload
description	Guide for uploading large files to ModelScope using Git LFS. Use when users need to upload large files, datasets, or models to ModelScope.
license	MIT

ModelScope Git LFS Upload Guide

About This Skill

This skill provides step-by-step instructions for uploading large files to ModelScope using Git LFS, including specific guidance for the test case provided.

When to Use

Use this skill when:

• Uploading large files (>100MB) to ModelScope
• Uploading datasets or models to ModelScope
• Following the test case requirements

Test Case Instructions

For the test case, follow these specific steps:

1. Clone the repository using the provided OAuth URL
2. Configure Git LFS for large file support
3. Add files to the repository
4. Push to ModelScope

Prerequisites

• Git and Git LFS installed
• ModelScope account
• Git access token from ModelScope

Step-by-Step Guide

1. Install Git LFS

git lfs install

2. Clone the Repository

Use the provided OAuth URL:

git clone https://oauth2:[e-mail]/datasets/syliubupt2025/ixi_bupt_mcx_shiyu.git

3. Configure Git LFS Tracking

# Track all large files
git lfs track "*.pt"  # For PyTorch models
git lfs track "*.pth"  # For PyTorch checkpoints
git lfs track "*.ckpt" # For general checkpoints
git lfs track "*.bin"  # For binary files
git lfs track "*.zip"  # For compressed files
git lfs track "*.tar.gz" # For compressed archives

# Track entire directories (optional)
git lfs track "data/**"  # Track all files in data directory
git lfs track "models/**" # Track all files in models directory

4. Add and Commit Files

# Add files
git add .

# Commit changes
git commit -m "Add large files"

5. Push to ModelScope

git push origin main

Test Case Specific Instructions

For the provided test case:

1. Clone the repository:

   git clone https://oauth2:[e-mail]/datasets/syliubupt2025/ixi_bupt_mcx_shiyu.git
   

2. Navigate to the repository:

   cd ixi_bupt_mcx_shiyu
   

3. Install Git LFS:

   git lfs install
   

4. Configure Git LFS tracking for your large files:

   git lfs track "*.pt" "*.pth" "*.ckpt" "*.bin" "*.zip" "*.tar.gz"
   

5. Add your large files to the repository:

   # Copy your large files here
   git add .
   

6. Commit and push:

   git commit -m "Add large files"
   git push origin main
   

Troubleshooting

Git LFS Authentication Issues

If you encounter authentication issues, ensure:

• Your OAuth token is correct and has the necessary permissions
• You're using the full OAuth URL format
• Git is configured with your credentials

Large File Pushing Issues

• Increase Git buffer size if pushing very large files:

  git config http.postBuffer 524288000  # 500MB buffer
  

• Check network connectivity if pushes fail

• Verify Git LFS is properly installed

Best Practices

• Track only large files with Git LFS (typically >100MB)
• Use .gitattributes to define LFS tracking rules
• Test with small files first before uploading large datasets
• Monitor push progress for large uploads
• Use descriptive commit messages for better version control

Output Example

When successful, you'll see output like:

Uploading LFS objects: 100% (3/3), 1.2 GB | 10.5 MB/s, done.
Enumerating objects: 15, done.
Counting objects: 100% (15/15), done.
Delta compression using up to 8 threads.
Compressing objects: 100% (12/12), done.
Writing objects: 100% (15/15), 2.15 KiB | 2.15 KiB/s, done.
Total 15 (delta 3), reused 0 (delta 0), pack-reused 0
To https://www.modelscope.cn/datasets/syliubupt2025/ixi_bupt_mcx_shiyu.git
   1a2b3c4..5d6e7f8  main -> main