Sample pack from YT video

I want to input a youtube video, then get a collection of music samples returned to me (.wav files in a zip). I think I should specify 3 specific times (eg 5 seconds, 130 seconds, 190 seconds, and then the agent rip the audio file from YouTube, then extract stems (instrumentals, vocals), and then extract (say) 16 beats worth of audio - beginning from the 3 timings I've entered. This gives us 3x separate audio sections to sample, and if we then separate them into vocals and instrumentals that then gives us 6x separate audio files to sample from (although we will use the initial tempo/beatpoint for all 6). Next we will want to use the beatpoint and tempo to extract samples (eg 1/2/4/8 beat in lengths, beginning from the beatpoints). So we chopup the samples eg using SOX, perhaps chopping based on beat length (if BPM = 90bpm, then 1 beat is 0.66secs) and beat position, and chopping maybe a mix of 1/2/4/8 beat samples. Finally we will want to save all the files to a server and zip them up.

First, pre-entered info...

final-output2

The final output of this process would be a zip file containing a curated selection of audio samples in .wav format. These samples would be extracted from a YouTube video at specified timestamps, separated into instrumental and vocal stems, and then chopped into various lengths (1, 2, 4, and 8 beats) based on the tempo and beat positions of the original audio. The zip file would contain the best 48 samples, filtered and ranked based on criteria such as average amplitude or other audio features.

subagent-summary-amended-for-actual-tasks

A) SUBAGENT SUMMARY: The SampleExtractorAgent subagent extracts multiple audio samples of varying lengths from instrumental and vocal stems based on tempo and beat position information. B) FINAL TASK OUTPUT: Multiple .wav files of varying lengths (1, 2, 4, or 8 beats) for each time point and stem type, extracted from the provided MP3 files using the tempo and beat position information. C) SUBAGENT INPUT: - 3 MP3 URLs of instrumental stems - Tempo and beat position information for each instrumental MP3 file E) SUBAGENT TASK SUMMARY: 1. Receive MP3 URLs of instrumental stems, and tempo and beat position information 2. For each instrumental MP3 file: a. Use skill #197 - Cut Wav/mp3 Audio into Multiple Pieces/Samples to extract 1, 2, 4, and 8 beat samples based on the tempo and beat positions 3. Output: List of .wav file URLs for the extracted samples 4. Output the final list of .wav file URLs F) SILOS: Silo 1: Instrumental Sample Extraction - Input: MP3 URLs of instrumental stems and their corresponding tempo and beat position information - Process: Use skill #197 to extract 1, 2, 4, and 8 beat samples for each instrumental MP3 file - Output: List of .wav file URLs for the extracted instrumental samples

subagentnumber

OVERVIEW OF SUBAGENTS: SUBAGENT 1: YouTube Video Processor - SKILL: #224 - Extract & Download Section Of YouTube Video URL - DESCRIPTION: This subagent will use the provided YouTube video URL and extract the audio from the three specified timestamp sections as MP4 files. - INPUT NEEDED: YouTube URL, start time (secs), and duration (secs) for each of the three sections - OUTPUT: Three MP4 URLs corresponding to the specified timestamp sections - SUBAGENT FINAL OUTPUT: [youtube-timestamp-mp4] SUBAGENT 2: Audio Converter - SKILL: #241 - Convert mp4 to mp3 - DESCRIPTION: This subagent will convert the extracted MP4 files to MP3 format for further processing. - INPUT NEEDED: MP4 URLs of the three timestamp sections - OUTPUT: Three MP3 URLs corresponding to the specified timestamp sections - SUBAGENT FINAL OUTPUT: [timestamp-mp3-audio] SUBAGENT 3: Stem Separator - SKILL: #204 - Extract Instrumental & Vocal From MP3 - DESCRIPTION: This subagent will separate the instrumental and vocal stems from the MP3 files obtained from the Audio Converter. - INPUT NEEDED: MP3 URLs of the three timestamp sections - OUTPUT: Six MP3 URLs (instrumental and vocal for each timestamp section) - SUBAGENT FINAL OUTPUT: [instrumental-vocal-stems] SUBAGENT 4: Beat Detector - SKILL: #195 - Extract Beatpoints & Tempo of MP3 - DESCRIPTION: This subagent will analyze the MP3 files to detect the tempo (BPM) and beat positions for each audio stem. - INPUT NEEDED: MP3 URLs of the instrumental and vocal stems for each timestamp section - OUTPUT: Tempo (BPM) and beat positions for each audio stem - SUBAGENT FINAL OUTPUT: [stem-beat-data] SUBAGENT 5: Audio Chopper - SKILL: #197 - Cut Wav/mp3 Audio into Multiple Pieces/Samples - DESCRIPTION: This subagent will chop the audio stems into 1, 2, 4, and 8 beat lengths based on the detected tempo and beat positions. - INPUT NEEDED: MP3 URLs of the instrumental and vocal stems, tempo (BPM), and beat positions for each timestamp section - OUTPUT: Multiple WAV files for each stem and timestamp section, representing the chopped samples of 1, 2, 4, and 8 beat lengths - SUBAGENT FINAL OUTPUT: [chopped-audio-samples] SUBAGENT 6: Sample Curator - SKILL: #196 - Create Visual Waveform Of Wav/mp3 File - SKILL: #198 - Analyze An Image With GPT Vision & Return Text - DESCRIPTION: This subagent will create visual waveforms of the chopped samples, analyze them using GPT Vision, and rank the samples based on criteria such as average amplitude. It will then select the top 48 samples across all stems and timestamp sections. - INPUT NEEDED: WAV files of the chopped samples for each stem and timestamp section - OUTPUT: 48 selected WAV files - SUBAGENT FINAL OUTPUT: [curated-48-samples] SUBAGENT 7: File Organizer - SKILL: #168 - Save text/html to server - DESCRIPTION: This subagent will apply a consistent naming convention and folder structure to the selected sample files. - INPUT NEEDED: 48 selected WAV files - OUTPUT: Consistently named and organized WAV files, ready for compression - SUBAGENT FINAL OUTPUT: [organized-sample-files] SUBAGENT 8: Zip Creator and Uploader - SKILL: #175 - Zip Internal URLs - DESCRIPTION: This subagent will compress the organized sample files into a zip file and upload it to a server for easy access and distribution. - INPUT NEEDED: Organized WAV files - OUTPUT: URL of the uploaded zip file containing the audio samples - SUBAGENT FINAL OUTPUT: [uploaded-sample-zip] UPDATED WORKFLOW: 1. User inputs YouTube video URL and three specific timestamps 2. YouTube Video Processor (#224) extracts audio from the specified timestamp sections as MP4 files 3. Audio Converter (#241) converts the extracted MP4 files to MP3 format 4. Stem Separator (#204) separates the instrumental and vocal stems from the MP3 files 5. Beat Detector (#195) analyzes the MP3 stems to detect tempo and beat positions 6. Audio Chopper (#197) chops the audio stems into 1, 2, 4, and 8 beat lengths based on the detected tempo and beat positions 7. Sample Curator (#196, #198) creates visual waveforms of the chopped samples, analyzes them using GPT Vision, ranks the samples, and selects the top 48 8. File Organizer (#168) applies a consistent naming convention and folder structure to the selected sample files 9. Zip Creator and Uploader (#175) compresses the organized sample files into a zip file and uploads it to a server for easy access and distribution ADDITIONAL NOTES: - The user interface for inputting the YouTube video URL and timestamps will be provided by the main agent. - The combination of the provided skills covers all the necessary functionality to complete the task, and no additional APIs or Python libraries are required.

Then, agent response...

subagent1refined-new-json-core

G) TASKFLOW SUMMARY Silo 1: Instrumental Sample Extraction - ID: #197 - Title task: Cut Wav/mp3 Audio into Multiple Pieces/Samples - Description of the task: Extract 1, 2, 4, and 8 beat samples from each instrumental MP3 file based on the provided tempo and beat position information using an audio processing library or tool. - Input(s) involved: [MP3 URLs of instrumental stems], [tempo and beat position information for each instrumental MP3 file] - Output token name: [List of .wav file URLs for the extracted instrumental samples]

subagent1mermaid

<!DOCTYPE html> <html> <head> <title>Mermaid Diagram Example</title> <script src="https://cdnjs.cloudflare.com/ajax/libs/mermaid/9.4.3/mermaid.min.js"></script> </head> <body> <div class="mermaid"> graph TD A[Receive MP3 URLs of instrumental stems, tempo, and beat positions] --> B{For each instrumental MP3 file} B --> C[Use skill #197 to extract 1 beat samples] B --> D[Use skill #197 to extract 2 beat samples] B --> E[Use skill #197 to extract 4 beat samples] B --> F[Use skill #197 to extract 8 beat samples] C --> G[Add extracted 1 beat samples to list] D --> H[Add extracted 2 beat samples to list] E --> I[Add extracted 4 beat samples to list] F --> J[Add extracted 8 beat samples to list] G --> K[Combine all extracted instrumental samples] H --> K I --> K J --> K K --> L[Output list of .wav file URLs for extracted instrumental samples] </div> <script> mermaid.initialize({ startOnLoad: true }); </script> </body> </html>

Mermaid Image - https://static.aiz.ac/1720012296-mermaid/mermaid1.png

prompts-required

Based on the provided taskflow summary, there is only one skill/task that requires a prompt: ID #: 197 Task: Cut Wav/mp3 Audio into Multiple Pieces/Samples Output of the task/skill: List of .wav file URLs for the extracted instrumental samples Description: For each of the provided MP3 URLs of instrumental stems, extract audio samples of 1, 2, 4, and 8 beat lengths based on the corresponding tempo and beat position information. Use an audio processing library or tool to precisely cut the audio files into the desired sample lengths. Ensure that the start of each sample aligns with the specified beat positions. Once all the samples have been extracted, generate a list of URLs pointing to the resulting .wav files. The output should be a comprehensive list containing the URLs of all the extracted instrumental samples.

subagent1-json-array

[ { "taskID": 1, "semanticTitleOfTask": "Extract Instrumental Samples", "taskDescription": "Extract 1, 2, 4, and 8 beat samples from each instrumental MP3 file based on the provided tempo and beat position information.", "inputDescription": "MP3 URLs of instrumental stems and their corresponding tempo and beat position information.", "inputRequired": ["instrumental-mp3-urls", "instrumental-tempo-beat-info"], "outputDescription": "A list of URLs pointing to the extracted .wav files of varying lengths (1, 2, 4, and 8 beats) for each instrumental MP3 file.", "outputName": "instrumental-sample-urls", "promptInstruction": "For each of the provided MP3 URLs of instrumental stems, extract audio samples of 1, 2, 4, and 8 beat lengths based on the corresponding tempo and beat position information. Use an audio processing library or tool to precisely cut the audio files into the desired sample lengths. Ensure that the start of each sample aligns with the specified beat positions. Once all the samples have been extracted, generate a list of URLs pointing to the resulting .wav files. The output should be a comprehensive list containing the URLs of all the extracted instrumental samples." } ]

subagent1-json-array-mermaid

<!DOCTYPE html> <html> <head> <title>Mermaid Diagram Example</title> <script src="https://cdnjs.cloudflare.com/ajax/libs/mermaid/9.4.3/mermaid.min.js"></script> </head> <body> <div class="mermaid"> graph TD A[Extract Instrumental Samples] --> B{Output: instrumental-sample-urls} B --> C[List of .wav file URLs for the extracted instrumental samples] classDef input fill:#F0F0F0,stroke:#333,stroke-width:1px; classDef output fill:#E0F0FF,stroke:#333,stroke-width:1px; classDef task fill:#FFFFFF,stroke:#333,stroke-width:1px; class A task class B output class C output </div> <script> mermaid.initialize({ startOnLoad: true }); </script> </body> </html>