Merged unrelated histories and resolved conflicts
This commit is contained in:
bnair
535
SETUP.md
535
SETUP.md
@@ -1,260 +1,169 @@
|
||||
# VMAF Optimisation Pipeline - Setup Guide
|
||||
|
||||
## Quick Start (Server)
|
||||
## Quick Start
|
||||
|
||||
### Prerequisites
|
||||
- FFmpeg with VMAF support (`ffmpeg -filters 2>&1 | grep libvmaf`)
|
||||
- Python 3.8+
|
||||
- FFmpeg with VMAF support (`ffmpeg -filters 2>&1 | grep libvmaf`)
|
||||
- ab-av1 binary (v0.10.3+)
|
||||
|
||||
### Installation
|
||||
|
||||
**On Linux (Server/WSL):**
|
||||
>>>>>>> 91418fa898de4a73e144a9fb9202b2315e922ab9
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone https://gitea.theflagroup.com/bnair/VMAFOptimiser.git /opt/Optmiser
|
||||
|
||||
# Download ab-av1 (if not present)
|
||||
# Download ab-av1
|
||||
cd /opt/Optmiser/bin
|
||||
wget https://github.com/alexheretic/ab-av1/releases/download/v0.10.3/ab-av1-x86_64-unknown-linux-musl
|
||||
chmod +x ab-av1
|
||||
|
||||
# Set up temporary directories
|
||||
# Set up directories
|
||||
mkdir -p /opt/Optmiser/tmp /opt/Optmiser/logs /opt/Optmiser/.lock
|
||||
```
|
||||
|
||||
### Basic Usage
|
||||
```bash
|
||||
# Process TV shows
|
||||
python3 /opt/Optmiser/optimise_media_v2.py /mnt/Media/tv tv_movies
|
||||
|
||||
# Process movies
|
||||
python3 /opt/Optmiser/optimise_media_v2.py /mnt/Media/movies tv_movies
|
||||
|
||||
# Process with 50% CPU limit (leaves CPU for other tasks)
|
||||
python3 /opt/Optmiser/optimise_media_v2.py /mnt/Media/tv tv_movies --cpu-limit 50
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Quick Start (Windows/WSL)
|
||||
|
||||
### WSL Installation (Recommended)
|
||||
```bash
|
||||
# Update WSL
|
||||
wsl --update
|
||||
|
||||
# Install dependencies in WSL Ubuntu/Debian
|
||||
sudo apt update
|
||||
sudo apt install -y ffmpeg python3 git
|
||||
|
||||
# Clone repository into WSL
|
||||
git clone https://gitea.theflagroup.com/bnair/VMAFOptimiser.git /opt/Optmiser
|
||||
|
||||
# Install ab-av1
|
||||
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
|
||||
source ~/.cargo/env
|
||||
cargo install ab-av1
|
||||
|
||||
# Access Windows media from WSL
|
||||
# Windows C:\ is at /mnt/c/
|
||||
python3 /opt/Optmiser/optimise_media_v2.py /mnt/c/Media/tv tv_movies
|
||||
```
|
||||
|
||||
### Native Windows Installation
|
||||
**On Windows:**
|
||||
```powershell
|
||||
# Install Python (if not present)
|
||||
# Download from python.org
|
||||
|
||||
# Install FFmpeg
|
||||
winget install ffmpeg
|
||||
|
||||
# Install Rust
|
||||
# Download from rust-lang.org
|
||||
|
||||
# Install ab-av1
|
||||
cargo install ab-av1
|
||||
|
||||
# Clone repository
|
||||
git clone https://gitea.theflagroup.com/bnair/VMAFOptimiser.git C:\Optmiser
|
||||
|
||||
# Run
|
||||
python C:\Optmiser\optimise_media_v2.py D:\Media\tv tv_movies
|
||||
# Install dependencies
|
||||
winget install ffmpeg
|
||||
|
||||
# Install Rust and ab-av1
|
||||
# Download from https://rust-lang.org/
|
||||
cargo install ab-av1
|
||||
|
||||
# Set up directories
|
||||
mkdir C:\Optmiser\tmp, C:\Optmiser\logs
|
||||
```
|
||||
|
||||
---
|
||||
**On macOS:**
|
||||
```bash
|
||||
# Clone repository
|
||||
git clone https://gitea.theflagroup.com/bnair/VMAFOptimiser.git /opt/Optmiser
|
||||
|
||||
## Running on Multiple Machines
|
||||
# Install dependencies
|
||||
brew install ffmpeg
|
||||
|
||||
### How Lock Files Work
|
||||
# Install Rust and ab-av1
|
||||
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
|
||||
cargo install ab-av1
|
||||
|
||||
Each video file has a corresponding lock file:
|
||||
# Set up directories
|
||||
mkdir -p /opt/Optmiser/tmp /opt/Optmiser/logs /opt/Optmiser/.lock
|
||||
```
|
||||
/opt/Optmiser/.lock/{video_filename}.lock
|
||||
|
||||
## Running the Optimiser
|
||||
|
||||
### Choose Your Script
|
||||
|
||||
**Linux / WSL / macOS:**
|
||||
```bash
|
||||
# Interactive mode (shows prompts)
|
||||
./run_optimisation.sh
|
||||
|
||||
# Direct execution with parameters
|
||||
./run_optimisation.sh --directory /mnt/Media/Movies --vmaf 95 --preset 6 --workers 1
|
||||
|
||||
# For 50% CPU mode on server
|
||||
./run_optimisation.sh --directory /mnt/Media/Movies --vmaf 95 --workers 1 --cpu-limit 50
|
||||
```
|
||||
|
||||
**Windows:**
|
||||
```powershell
|
||||
# Interactive mode (shows prompts)
|
||||
.\run_optimisation.ps1
|
||||
|
||||
# Direct execution with parameters
|
||||
.\run_optimisation.ps1 -directory "D:\Movies" --vmaf 95 --preset 6 --workers 1
|
||||
|
||||
# For hardware acceleration (AMD GPU)
|
||||
.\run_optimisation.ps1 -directory "D:\Movies" --vmaf 95 --hwaccel auto
|
||||
```
|
||||
|
||||
## Script Parameters
|
||||
|
||||
All wrapper scripts (`run_optimisation.sh` on Linux, `run_optimisation.ps1` on Windows) accept these parameters:
|
||||
|
||||
| Parameter | Description | Default |
|
||||
|------------|-------------|---------|
|
||||
| `--directory <path>` | Root directory to scan | Current directory |
|
||||
| `--vmaf <score>` | Target VMAF score | 95.0 |
|
||||
| `--preset <value>` | SVT-AV1 Preset (4=best, 6=balanced, 8=fast) | 6 |
|
||||
| `--workers <count>` | Concurrent files to process | 1 |
|
||||
| `--samples <count>` | Samples for CRF search | 4 |
|
||||
| `--thorough` | Use thorough mode (slower, more accurate) | false |
|
||||
| `--encoder <name>` | ab-av1 encoder | svt-av1 |
|
||||
| `--hwaccel <value>` | Hardware acceleration | none (auto: auto-detect) |
|
||||
|
||||
## Multi-Machine Setup
|
||||
|
||||
### How Lock Files Prevent Conflicts
|
||||
|
||||
Each video file has a lock file: `/opt/Optmiser/.lock/{video_filename}`
|
||||
|
||||
**Process:**
|
||||
1. Machine A checks for lock → None found, creates lock
|
||||
2. Machine A starts encoding
|
||||
3. Machine B checks for lock → Found, skips file
|
||||
4. Machine A finishes, removes lock
|
||||
5. Machine B can now process that file
|
||||
1. Machine A: Checks for lock → Not found, creates lock
|
||||
2. Machine A: Starts encoding
|
||||
3. Machine B: Checks for lock → Found, skips file
|
||||
4. Machine A: Finishes, removes lock
|
||||
5. Machine B: Can now process that file
|
||||
|
||||
**Result:** Different machines automatically process different files!
|
||||
**Result:** Different machines automatically process different files simultaneously!
|
||||
|
||||
### Setup for Multiple Machines
|
||||
|
||||
**Machine 1 - Remote Server (Intel i9-12900H):**
|
||||
**Machine 1 - Linux Server (Intel i9-12900H):**
|
||||
```bash
|
||||
cd /opt/Optmiser
|
||||
git pull origin main
|
||||
|
||||
# Run on /mnt/Media (Linux filesystem)
|
||||
python3 optimise_media_v2.py /mnt/Media/tv tv_movies
|
||||
python3 optimise_media_v2.py /mnt/Media/movies tv_movies
|
||||
|
||||
# With 50% CPU limit (recommended)
|
||||
python3 optimise_media_v2.py /mnt/Media/tv tv_movies --cpu-limit 50
|
||||
./run_optimisation.sh /mnt/Media/movies --vmaf 95
|
||||
```
|
||||
|
||||
**Machine 2 - Local PC (AMD RX 7900 XT, Windows):**
|
||||
|
||||
Option A - Native Windows:
|
||||
**Machine 2 - Windows PC (AMD RX 7900 XT):**
|
||||
```powershell
|
||||
# Map network drive if needed
|
||||
# \\Server\media\ or use local storage
|
||||
|
||||
cd C:\Optmiser
|
||||
git pull origin main
|
||||
|
||||
# Run on local media
|
||||
python optimise_media_v2.py D:\Media\tv tv_movies
|
||||
.\run_optimisation.ps1 D:\Media\movies --vmaf 95 --hwaccel auto
|
||||
```
|
||||
|
||||
Option B - WSL (Recommended):
|
||||
```bash
|
||||
# Windows C: drive accessible at /mnt/c/
|
||||
cd /opt/Optmiser
|
||||
python optimise_media_v2.py /mnt/c/Media/tv tv_movies
|
||||
```
|
||||
|
||||
**Machine 3 - Another PC (AMD 9800X3D, Linux):**
|
||||
**Machine 3 - Another Linux PC:**
|
||||
```bash
|
||||
cd /opt/Optmiser
|
||||
git pull origin main
|
||||
|
||||
# Run on local media directory
|
||||
python optimise_media_v2.py /home/user/Media/tv tv_movies
|
||||
./run_optimisation.sh /home/user/Media/tv --vmaf 95
|
||||
```
|
||||
|
||||
All three can run simultaneously - lock files prevent duplicates!
|
||||
All three can run simultaneously!
|
||||
|
||||
---
|
||||
## Hardware Acceleration
|
||||
|
||||
## Hardware Encoding Guide
|
||||
### Automatic Detection
|
||||
|
||||
### Detecting Hardware
|
||||
When `--hwaccel auto` is specified, the wrapper scripts automatically select the best available hardware acceleration:
|
||||
|
||||
The script automatically checks for:
|
||||
1. **AMD GPU encoding support** (future feature)
|
||||
2. **System thread count**
|
||||
3. **Operating system** (Linux vs Windows)
|
||||
| Platform | Auto Selection | Notes |
|
||||
|-----------|----------------|--------|
|
||||
| Windows | d3d11va | Direct3D Video Acceleration |
|
||||
| macOS | videotoolbox | VideoToolbox framework |
|
||||
| Linux/WSL | vaapi | Video Acceleration via VA-API |
|
||||
|
||||
### CPU Encoding (Software)
|
||||
### GPU vs iGPU Priority
|
||||
|
||||
**Best for:** Servers, multi-tasking
|
||||
|
||||
```bash
|
||||
# Force CPU encoding
|
||||
python3 optimise_media_v2.py /media --use-cpu
|
||||
|
||||
# Limit to 50% of CPU (12 threads on 24-core)
|
||||
python3 optimise_media_v2.py /media --cpu-limit 50
|
||||
|
||||
# Set specific worker count
|
||||
python3 optimise_media_v2.py /media --workers 8
|
||||
```
|
||||
|
||||
**When to use:**
|
||||
- Running Plex/Jellyfin on same machine
|
||||
- Server environment with multiple users
|
||||
- Want to leave GPU free for other tasks
|
||||
|
||||
**Expected Speed:** 8-15 fps @ 1080p
|
||||
|
||||
### GPU Encoding (AMD RX 7900 XT)
|
||||
|
||||
**Status:** Planned for future versions
|
||||
**Expected Speedup:** 3-10x vs CPU encoding
|
||||
|
||||
```bash
|
||||
# Enable hardware encoding (when implemented)
|
||||
python3 optimise_media_v2.py /media --use-hardware
|
||||
```
|
||||
|
||||
**When to use:**
|
||||
- Local PC with AMD GPU
|
||||
- Want faster encoding speeds
|
||||
- Can compensate quality with slightly lower VMAF target
|
||||
|
||||
**Expected Speed:** 150+ fps @ 1080p
|
||||
|
||||
### Quality Compensation for GPU Encoding
|
||||
|
||||
GPU encoding may need quality adjustments:
|
||||
|
||||
| CPU VMAF | Equivalent GPU VMAF | Why |
|
||||
|------------|----------------------|-----|
|
||||
| 94 | 92-93 | GPU has quality limitations |
|
||||
| 93 | 91-92 | Hardware encoding trade-offs |
|
||||
| 90 | 88-89 | Significant compression |
|
||||
|
||||
**Recommendation:** If using GPU encoding, set `TARGETS = [92.0, 90.0, 88.0]` for similar quality.
|
||||
|
||||
---
|
||||
|
||||
## Windows/WSL Path Mapping
|
||||
|
||||
### Understanding /mnt/c/
|
||||
|
||||
In WSL, Windows drives are mapped:
|
||||
|
||||
| Windows | WSL Path | Example |
|
||||
|---------|------------|---------|
|
||||
| C:\ | /mnt/c/ | /mnt/c/Users/bnair/Downloads |
|
||||
| D:\ | /mnt/d/ | /mnt/d/Movies/ |
|
||||
| E:\ | /mnt/e/ | /mnt/e/TV\ |
|
||||
|
||||
**To access Windows media from WSL:**
|
||||
```bash
|
||||
# List C:\Media\tv
|
||||
ls /mnt/c/Media/tv
|
||||
|
||||
# Process from WSL
|
||||
python3 /opt/Optmiser/optimise_media_v2.py /mnt/c/Media/tv tv_movies
|
||||
```
|
||||
|
||||
### Network Drives on WSL
|
||||
|
||||
```bash
|
||||
# Map network drive (one-time)
|
||||
sudo mkdir -p /mnt/server
|
||||
echo "//192.168.1.100/media | /mnt/server cifs credentials=/path/to/credfile,uid=1000,gid=1000 0 0" | sudo tee -a /etc/fstab
|
||||
|
||||
# Access network media
|
||||
python3 /opt/Optmiser/optimise_media_v2.py /mnt/server/Media/tv tv_movies
|
||||
```
|
||||
|
||||
---
|
||||
- **Discrete GPU takes priority:** If a discrete GPU (like AMD RX 7900 XT) is present, it's selected over integrated GPU
|
||||
- **For AMD RX 7900 XT:** Hardware encoding provides ~3-10x speedup over CPU
|
||||
- **Note:** GPU encoding may need slightly lower VMAF targets to match CPU quality
|
||||
|
||||
## Customization
|
||||
|
||||
### Changing VMAF Targets
|
||||
|
||||
Edit `optimise_media_v2.py`:
|
||||
Edit `optimize_library.py`:
|
||||
|
||||
```python
|
||||
# Line 15
|
||||
TARGETS = [94.0, 93.0, 92.0, 90.0]
|
||||
|
||||
# More aggressive (smaller files, lower quality)
|
||||
TARGETS = [92.0, 90.0, 88.0]
|
||||
|
||||
@@ -265,18 +174,16 @@ TARGETS = [95.0, 94.0, 93.0]
|
||||
### Changing Savings Threshold
|
||||
|
||||
```python
|
||||
# Line 17
|
||||
MIN_SAVINGS_PERCENT = 12.0 # Current threshold
|
||||
MIN_SAVINGS_PERCENT = 8.0 # More aggressive (encode more)
|
||||
MIN_SAVINGS_PERCENT = 15.0 # Less aggressive (encode fewer)
|
||||
# More aggressive (encode more)
|
||||
MIN_SAVINGS_PERCENT = 8.0
|
||||
|
||||
# Less aggressive (encode fewer)
|
||||
MIN_SAVINGS_PERCENT = 15.0
|
||||
```
|
||||
|
||||
### Changing Encoder Preset
|
||||
|
||||
```python
|
||||
# Line 19
|
||||
PRESET = 6
|
||||
|
||||
# Faster encodes (larger files, lower quality)
|
||||
PRESET = 8
|
||||
|
||||
@@ -284,116 +191,112 @@ PRESET = 8
|
||||
PRESET = 4
|
||||
```
|
||||
|
||||
### Changing Estimate Target
|
||||
## Platform-Specific Tips
|
||||
|
||||
```python
|
||||
# Line 18
|
||||
TARGET_SAVINGS_FOR_ESTIMATE = 15.0
|
||||
### For Linux Servers (Intel i9-12900H)
|
||||
|
||||
# Target higher savings
|
||||
TARGET_SAVINGS_FOR_ESTIMATE = 20.0
|
||||
```
|
||||
1. **Use 50% CPU mode** if running other services:
|
||||
```bash
|
||||
./run_optimisation.sh --directory /media --vmaf 95 --workers 1 --cpu-limit 50
|
||||
```
|
||||
|
||||
---
|
||||
2. **Run during off-peak hours** to minimize user impact
|
||||
|
||||
## Monitoring
|
||||
3. **Monitor CPU temperature:**
|
||||
```bash
|
||||
watch -n 2 'sensors | grep "Package id"'
|
||||
```
|
||||
|
||||
### Watch Progress in Real-Time
|
||||
4. **Use higher preset for faster encodes:**
|
||||
```bash
|
||||
./run_optimisation.sh --vmaf 93 --preset 8
|
||||
```
|
||||
|
||||
### For Windows PCs (AMD RX 7900 XT)
|
||||
|
||||
1. **Enable hardware acceleration** for massive speedup:
|
||||
```powershell
|
||||
.\run_optimisation.ps1 --directory "D:\Movies" --hwaccel auto
|
||||
```
|
||||
|
||||
2. **Test small sample first** to verify settings:
|
||||
```powershell
|
||||
.\run_optimisation.ps1 --directory "D:\Media\sample" --thorough --vmaf 95
|
||||
```
|
||||
|
||||
3. **Monitor GPU usage** (Task Manager or third-party tools)
|
||||
|
||||
4. **Consider quality compensation** - GPU encoding may need slightly lower VMAF targets to match CPU quality
|
||||
|
||||
### For WSL (Ubuntu/Debian)
|
||||
|
||||
1. **Access Windows drives** via `/mnt/c/`:
|
||||
```bash
|
||||
ls /mnt/c/Media/movies
|
||||
```
|
||||
|
||||
2. **Increase memory limits** if encoding 4K content:
|
||||
```bash
|
||||
# Edit ~/.wslconfig
|
||||
[wsl2]
|
||||
memory=16GB
|
||||
```
|
||||
|
||||
## Running in Docker (Optional)
|
||||
|
||||
```bash
|
||||
# Tail log file with JSON formatting
|
||||
tail -f /opt/Optmiser/logs/tv_movies.jsonl | jq '.'
|
||||
# Build image
|
||||
docker build -t vmaf-optimiser .
|
||||
|
||||
# Monitor encoding speed
|
||||
watch -n 5 'jq -r "[.savings, .duration]" /opt/Optmiser/logs/tv_movies.jsonl | tail -1'
|
||||
|
||||
# Check lock files (what's being processed)
|
||||
ls -la /opt/Optmiser/.lock/
|
||||
# Run with mount
|
||||
docker run -v /path/to/media:/media vmaf-optimiser /media
|
||||
```
|
||||
|
||||
### Performance Dashboard
|
||||
|
||||
```bash
|
||||
# Create a simple dashboard
|
||||
watch -n 10 '
|
||||
echo "=== VMAF Optimiser Status ==="
|
||||
echo ""
|
||||
echo "Recent Encodes:"
|
||||
tail -3 /opt/Optmiser/logs/tv_movies.jsonl | jq -r "[.file, .savings, .duration] | @tsv"
|
||||
echo ""
|
||||
echo "CPU Usage:"
|
||||
top -bn1 | head -5
|
||||
'
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Issue: Scripts not found
|
||||
|
||||
**Solution:** Ensure you're in the correct directory with the scripts installed.
|
||||
|
||||
### Issue: "ab-av1: command not found"
|
||||
|
||||
**Solution:** Install ab-av1 via cargo:
|
||||
```bash
|
||||
# Check path
|
||||
ls /opt/Optmiser/optimise_media_v2.py
|
||||
|
||||
# Check Python version
|
||||
python3 --version # Should be 3.8+
|
||||
|
||||
# Check ab-av1
|
||||
/opt/Optmiser/bin/ab-av1 --version # Should be 0.10.3+
|
||||
```
|
||||
|
||||
### Issue: Permission denied
|
||||
|
||||
```bash
|
||||
# Make scripts executable
|
||||
chmod +x /opt/Optmiser/*.py
|
||||
chmod +x /opt/Optmiser/*.sh
|
||||
|
||||
# Fix lock directory permissions
|
||||
chmod 777 /opt/Optmiser/.lock
|
||||
```
|
||||
|
||||
### Issue: ab-av1 not found
|
||||
|
||||
```bash
|
||||
# Check if in PATH
|
||||
which ab-av1
|
||||
|
||||
# Use full path if not in PATH
|
||||
/opt/Optmiser/bin/ab-av1 --version
|
||||
|
||||
# Add to PATH
|
||||
export PATH="$PATH:/opt/Optmiser/bin"
|
||||
cargo install ab-av1
|
||||
```
|
||||
|
||||
### Issue: FFmpeg VMAF not available
|
||||
|
||||
```bash
|
||||
# Check libvmaf
|
||||
ffmpeg -filters 2>&1 | grep libvmaf
|
||||
|
||||
# If not found, recompile FFmpeg with VMAF
|
||||
# Or download compiled version from johnvansickle.com/ffmpeg
|
||||
```
|
||||
**Solution:** Recompile FFmpeg with VMAF support or download a pre-built version that includes libvmaf.
|
||||
|
||||
### Issue: Out of Memory
|
||||
|
||||
**Solution:** Reduce workers or increase swap:
|
||||
```bash
|
||||
# Check available memory
|
||||
free -h
|
||||
|
||||
# Reduce workers
|
||||
python3 optimise_media_v2.py /media --workers 4
|
||||
|
||||
# Increase swap (if needed)
|
||||
# Increase swap
|
||||
sudo fallocate -l 4G /swapfile
|
||||
sudo chmod 600 /swapfile
|
||||
sudo mkswap /swapfile
|
||||
sudo swapon /swapfile
|
||||
|
||||
# Use more conservative settings
|
||||
./run_optimisation.sh --workers 1 --vmaf 93
|
||||
```
|
||||
|
||||
---
|
||||
### Issue: Multiple machines encoding same file
|
||||
|
||||
**Solution:** This is prevented by lock files. If you see duplicates, check `/opt/Optmiser/.lock/` for stale locks.
|
||||
|
||||
### Issue: Encoding fails
|
||||
|
||||
**Solution:** Check logs:
|
||||
```bash
|
||||
cat /opt/Optmiser/logs/failed_encodes.jsonl | jq '.'
|
||||
```
|
||||
|
||||
### Issue: "unexpected argument" error
|
||||
|
||||
**Solution:** Use correct flags for your ab-av1 version. The wrapper scripts now validate support at runtime.
|
||||
|
||||
## Git Workflow
|
||||
|
||||
@@ -414,74 +317,90 @@ cd /opt/Optmiser
|
||||
git pull origin main
|
||||
|
||||
# Run optimisation
|
||||
python3 optimise_media_v2.py /media tv_movies
|
||||
./run_optimisation.sh /media tv_movies
|
||||
|
||||
# Review changes
|
||||
git diff
|
||||
```
|
||||
|
||||
### Commit Changes
|
||||
### Committing Changes
|
||||
|
||||
```bash
|
||||
cd /opt/Optmiser
|
||||
git status
|
||||
|
||||
# Add changed files
|
||||
git add optimise_media_v2.py run_optimisation.sh
|
||||
git add optimize_library.py run_optimisation.sh run_optimisation.ps1
|
||||
|
||||
# Commit with message
|
||||
git commit -m "feat: add X"
|
||||
git commit -m "feat: add Windows and Linux wrapper scripts"
|
||||
|
||||
# Push
|
||||
git push
|
||||
```
|
||||
|
||||
### Viewing Logs
|
||||
|
||||
```bash
|
||||
# Watch logs in real-time
|
||||
tail -f /opt/Optmiser/logs/tv_movies.jsonl | jq '.'
|
||||
|
||||
# Check files logged for review
|
||||
cat /opt/Optmiser/logs/low_savings_skips.jsonl | jq '.[] | select(.recommendations=="logged_for_review")'
|
||||
|
||||
# Statistics
|
||||
jq -r '.status' /opt/Optmiser/logs/tv_movies.jsonl | sort | uniq -c
|
||||
|
||||
# Find what CRF/VMAF combinations are being used most
|
||||
jq -r '[.vmaf, .crf] | @tsv' /opt/Optmiser/logs/tv_movies.jsonl | sort | uniq -c
|
||||
```
|
||||
|
||||
### View History
|
||||
|
||||
```bash
|
||||
cd /opt/Optmiser
|
||||
git log --oneline
|
||||
git log --graph --all
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## FAQ
|
||||
|
||||
**Q: Can I run this on multiple machines at once?**
|
||||
A: Yes! Each machine will process different files due to lock file mechanism.
|
||||
|
||||
**Q: Will hardware encoding be available?**
|
||||
A: Planned for future versions. Currently uses CPU encoding (software AV1).
|
||||
**Q: Should I use Windows or WSL?**
|
||||
A: WSL is recommended for Linux compatibility. Use Windows native if you need direct hardware access or performance.
|
||||
|
||||
**Q: How do I know if a file is being encoded elsewhere?**
|
||||
A: Check `/opt/Optmiser/.lock/` - if a lock exists, another machine is processing it.
|
||||
**Q: Will hardware encoding work better than CPU?**
|
||||
A: For AMD RX 7900 XT, hardware AV1 encoding is ~3-10x faster than CPU. However, GPU encoding may need slightly lower VMAF targets to match CPU quality.
|
||||
|
||||
**Q: Can I change the VMAF target?**
|
||||
A: Yes, edit `TARGETS = [94.0, 93.0, 92.0, 90.0]` in `optimise_media_v2.py`.
|
||||
**Q: What VMAF target should I use?**
|
||||
A: Start with VMAF 94 or 95. Drop to 92-90 if you need more savings.
|
||||
|
||||
**Q: What if savings are always below 12%?**
|
||||
A: The script will log 15% estimates to `low_savings_skips.jsonl`. Review these logs to decide if encoding is worth it.
|
||||
|
||||
**Q: Does this work on Windows/WSL?**
|
||||
A: Yes! See the Windows/WSL section for setup instructions.
|
||||
|
||||
**Q: How much CPU does encoding use?**
|
||||
A: Full CPU (24 threads) by default. Use `--cpu-limit 50` for 50% mode.
|
||||
**Q: How do I know which files are being processed?**
|
||||
A: Check `.lock/` directory: `ls -la /opt/Optmiser/.lock/`
|
||||
|
||||
**Q: Can I pause/resume?**
|
||||
A: Pause by stopping the script (Ctrl+C). Resume by running again - it skips processed files.
|
||||
|
||||
**Q: What happens if encoding fails?**
|
||||
A: Error is logged to `failed_encodes.jsonl` with error code. Original file is NOT modified.
|
||||
A: Error is logged to `failed_encodes.jsonl`. Original file is NOT modified.
|
||||
|
||||
---
|
||||
**Q: How much CPU does encoding use?**
|
||||
A: Full CPU by default. Use `--workers 1` for single-threaded, or limit with `--cpu-limit 50` for 50% (12 threads on 24-core).
|
||||
|
||||
## Support
|
||||
**Q: What are the log files?**
|
||||
A:
|
||||
- `tv_movies.jsonl` - Successful TV & Movie encodes
|
||||
- `content.jsonl` - Successful Content folder encodes
|
||||
- `low_savings_skips.jsonl` - Files with <12% savings + 15% estimates
|
||||
- `failed_searches.jsonl` - Files that couldn't hit any VMAF target
|
||||
- `failed_encodes.jsonl` - Encoding errors
|
||||
|
||||
For issues or questions:
|
||||
- Check AGENTS.md for detailed technical documentation
|
||||
- Review logs in `/opt/Optmiser/logs/`
|
||||
- Test with `--dry-run` flag first
|
||||
**Q: How do I see real-time progress?**
|
||||
A: The script streams all ab-av1 output in real-time, showing ETA, encoding speed, and frame count.
|
||||
|
||||
---
|
||||
|
||||
**Last Updated:** December 31, 2025
|
||||
**Version:** 2.0 with Windows and Linux Wrapper Scripts
|
||||
|
||||
Reference in New Issue
Block a user