Sample pack from YT video

I want to input a youtube video, then get a collection of music samples returned to me (.wav files in a zip). I think I should specify 3 specific times (eg 5 seconds, 130 seconds, 190 seconds, and then the agent rip the audio file from YouTube, then extract stems (instrumentals, vocals), and then extract (say) 16 beats worth of audio - beginning from the 3 timings I've entered. This gives us 3x separate audio sections to sample, and if we then separate them into vocals and instrumentals that then gives us 6x separate audio files to sample from (although we will use the initial tempo/beatpoint for all 6). Next we will want to use the beatpoint and tempo to extract samples (eg 1/2/4/8 beat in lengths, beginning from the beatpoints). So we chopup the samples eg using SOX, perhaps chopping based on beat length (if BPM = 90bpm, then 1 beat is 0.66secs) and beat position, and chopping maybe a mix of 1/2/4/8 beat samples. Finally we will want to save all the files to a server and zip them up.

First, pre-entered info...

final-output2

The final output of this process would be a zip file containing a curated selection of audio samples in .wav format. These samples would be extracted from a YouTube video at specified timestamps, separated into instrumental and vocal stems, and then chopped into various lengths (1, 2, 4, and 8 beats) based on the tempo and beat positions of the original audio. The zip file would contain the best 48 samples, filtered and ranked based on criteria such as average amplitude or other audio features.

subagent-summary-amended-for-actual-tasks

A) SUBAGENT SUMMARY: StemSeparatorAgent separates MP3 audio files into instrumental and vocal stems for each time point. B) FINAL TASK OUTPUT: Six MP3 files containing separated instrumental and vocal stems for three specific time points in the original YouTube video. C) SUBAGENT INPUT: Three MP3 files, one for each time point extracted from the YouTube video by AudioExtractorAgent. E) SUBAGENT TASK SUMMARY: - Input: Three MP3 files from AudioExtractorAgent - Skill #204 - Extract Instrumental & Vocal From MP3 (run 3 times, once for each time point MP3) - Input: MP3 URL - Output: Two MP3 URLs per input (instrumental and vocal stems) - Output: Six total MP3 files (instrumental and vocal stems for each of the three time points) The specific flow would be: MP3 URLs from AudioExtractorAgent > #204 (time point 1) > #204 (time point 2) > #204 (time point 3) > Six MP3 URLs (separated audio stems) F) SILOS: This subagent does not require silos as it performs the same task (separating instrumental and vocal stems) for each of the three time point MP3 files in sequence. So in summary, the StemSeparatorAgent takes the three MP3 files from specific time points in the YouTube video provided by the AudioExtractorAgent, and runs skill #204 three times (once for each MP3) to separate the instrumental and vocal stems. This results in a total of six output MP3 files - an instrumental and vocal stem for each of the three time points. These files are then ready to be passed on to the next subagent for further processing.

subagentnumber

OVERVIEW OF SUBAGENTS: SUBAGENT 1: YouTube Video Processor - SKILL: #224 - Extract & Download Section Of YouTube Video URL - DESCRIPTION: This subagent will use the provided YouTube video URL and extract the audio from the three specified timestamp sections as MP4 files. - INPUT NEEDED: YouTube URL, start time (secs), and duration (secs) for each of the three sections - OUTPUT: Three MP4 URLs corresponding to the specified timestamp sections - SUBAGENT FINAL OUTPUT: [youtube-timestamp-mp4] SUBAGENT 2: Audio Converter - SKILL: #241 - Convert mp4 to mp3 - DESCRIPTION: This subagent will convert the extracted MP4 files to MP3 format for further processing. - INPUT NEEDED: MP4 URLs of the three timestamp sections - OUTPUT: Three MP3 URLs corresponding to the specified timestamp sections - SUBAGENT FINAL OUTPUT: [timestamp-mp3-audio] SUBAGENT 3: Stem Separator - SKILL: #204 - Extract Instrumental & Vocal From MP3 - DESCRIPTION: This subagent will separate the instrumental and vocal stems from the MP3 files obtained from the Audio Converter. - INPUT NEEDED: MP3 URLs of the three timestamp sections - OUTPUT: Six MP3 URLs (instrumental and vocal for each timestamp section) - SUBAGENT FINAL OUTPUT: [instrumental-vocal-stems] SUBAGENT 4: Beat Detector - SKILL: #195 - Extract Beatpoints & Tempo of MP3 - DESCRIPTION: This subagent will analyze the MP3 files to detect the tempo (BPM) and beat positions for each audio stem. - INPUT NEEDED: MP3 URLs of the instrumental and vocal stems for each timestamp section - OUTPUT: Tempo (BPM) and beat positions for each audio stem - SUBAGENT FINAL OUTPUT: [stem-beat-data] SUBAGENT 5: Audio Chopper - SKILL: #197 - Cut Wav/mp3 Audio into Multiple Pieces/Samples - DESCRIPTION: This subagent will chop the audio stems into 1, 2, 4, and 8 beat lengths based on the detected tempo and beat positions. - INPUT NEEDED: MP3 URLs of the instrumental and vocal stems, tempo (BPM), and beat positions for each timestamp section - OUTPUT: Multiple WAV files for each stem and timestamp section, representing the chopped samples of 1, 2, 4, and 8 beat lengths - SUBAGENT FINAL OUTPUT: [chopped-audio-samples] SUBAGENT 6: Sample Curator - SKILL: #196 - Create Visual Waveform Of Wav/mp3 File - SKILL: #198 - Analyze An Image With GPT Vision & Return Text - DESCRIPTION: This subagent will create visual waveforms of the chopped samples, analyze them using GPT Vision, and rank the samples based on criteria such as average amplitude. It will then select the top 48 samples across all stems and timestamp sections. - INPUT NEEDED: WAV files of the chopped samples for each stem and timestamp section - OUTPUT: 48 selected WAV files - SUBAGENT FINAL OUTPUT: [curated-48-samples] SUBAGENT 7: File Organizer - SKILL: #168 - Save text/html to server - DESCRIPTION: This subagent will apply a consistent naming convention and folder structure to the selected sample files. - INPUT NEEDED: 48 selected WAV files - OUTPUT: Consistently named and organized WAV files, ready for compression - SUBAGENT FINAL OUTPUT: [organized-sample-files] SUBAGENT 8: Zip Creator and Uploader - SKILL: #175 - Zip Internal URLs - DESCRIPTION: This subagent will compress the organized sample files into a zip file and upload it to a server for easy access and distribution. - INPUT NEEDED: Organized WAV files - OUTPUT: URL of the uploaded zip file containing the audio samples - SUBAGENT FINAL OUTPUT: [uploaded-sample-zip] UPDATED WORKFLOW: 1. User inputs YouTube video URL and three specific timestamps 2. YouTube Video Processor (#224) extracts audio from the specified timestamp sections as MP4 files 3. Audio Converter (#241) converts the extracted MP4 files to MP3 format 4. Stem Separator (#204) separates the instrumental and vocal stems from the MP3 files 5. Beat Detector (#195) analyzes the MP3 stems to detect tempo and beat positions 6. Audio Chopper (#197) chops the audio stems into 1, 2, 4, and 8 beat lengths based on the detected tempo and beat positions 7. Sample Curator (#196, #198) creates visual waveforms of the chopped samples, analyzes them using GPT Vision, ranks the samples, and selects the top 48 8. File Organizer (#168) applies a consistent naming convention and folder structure to the selected sample files 9. Zip Creator and Uploader (#175) compresses the organized sample files into a zip file and uploads it to a server for easy access and distribution ADDITIONAL NOTES: - The user interface for inputting the YouTube video URL and timestamps will be provided by the main agent. - The combination of the provided skills covers all the necessary functionality to complete the task, and no additional APIs or Python libraries are required.

Then, agent response...

subagent1refined-new-json-core

G) TASKFLOW SUMMARY - ID: #204 - Title task: Extract Instrumental & Vocal From MP3 - Description of the task: Separates an input MP3 file into two separate MP3 files - one containing only the instrumental track and the other containing only the vocal track. - Input(s) involved: [MP3 URL] - Output token name: [Instrumental MP3 URL], [Vocal MP3 URL] The StemSeparatorAgent uses skill #204 three times in sequence, once for each time point MP3 file provided by the AudioExtractorAgent. There are no additional skills or silos involved in this subagent.

subagent1mermaid

<!DOCTYPE html> <html> <head> <title>Mermaid Diagram Example</title> <script src="https://cdnjs.cloudflare.com/ajax/libs/mermaid/9.4.3/mermaid.min.js"></script> </head> <body> <div class="mermaid"> graph TD A[AudioExtractorAgent MP3 URLs] --> B{StemSeparatorAgent} B --> C[Skill #204 - Extract Instrumental & Vocal From MP3] C --> D[Time Point 1 MP3 URL] C --> E[Time Point 2 MP3 URL] C --> F[Time Point 3 MP3 URL] D --> G[Instrumental MP3 URL 1] D --> H[Vocal MP3 URL 1] E --> I[Instrumental MP3 URL 2] E --> J[Vocal MP3 URL 2] F --> K[Instrumental MP3 URL 3] F --> L[Vocal MP3 URL 3] G --> M[Six Total MP3 Files] H --> M I --> M J --> M K --> M L --> M </div> <script> mermaid.initialize({ startOnLoad: true }); </script> </body> </html>

Mermaid Image - https://static.aiz.ac/1720012277-mermaid/mermaid1.png

prompts-required

Based on the provided taskflow summary, the StemSeparatorAgent does not require any LLM prompts. The agent uses skill #204 (Extract Instrumental & Vocal From MP3) three times in sequence, once for each time point MP3 file provided by the AudioExtractorAgent. Skill #204 takes an MP3 URL as input and outputs two separate MP3 URLs - one for the instrumental track and another for the vocal track. This task is likely performed using a pre-existing software or API that can separate the vocal and instrumental components of an audio file without the need for an LLM prompt. Therefore, no unique prompts need to be written for the skills involved in the StemSeparatorAgent workflow.

subagent1-json-array

[ { "taskID": 204, "semanticTitleOfTask": "Extract Instrumental & Vocal From MP3 - Time Point 1", "taskDescription": "Separates the first time point MP3 file into two separate MP3 files - one containing only the instrumental track and the other containing only the vocal track.", "inputDescription": "The MP3 URL of the first time point extracted from the YouTube video by AudioExtractorAgent.", "inputRequired": ["time-point-1-mp3-url"], "outputDescription": "Two MP3 URLs - one for the instrumental track and another for the vocal track of the first time point.", "outputName": ["time-point-1-instrumental-mp3-url", "time-point-1-vocal-mp3-url"], "promptInstruction": "" }, { "taskID": 204, "semanticTitleOfTask": "Extract Instrumental & Vocal From MP3 - Time Point 2", "taskDescription": "Separates the second time point MP3 file into two separate MP3 files - one containing only the instrumental track and the other containing only the vocal track.", "inputDescription": "The MP3 URL of the second time point extracted from the YouTube video by AudioExtractorAgent.", "inputRequired": ["time-point-2-mp3-url"], "outputDescription": "Two MP3 URLs - one for the instrumental track and another for the vocal track of the second time point.", "outputName": ["time-point-2-instrumental-mp3-url", "time-point-2-vocal-mp3-url"], "promptInstruction": "" }, { "taskID": 204, "semanticTitleOfTask": "Extract Instrumental & Vocal From MP3 - Time Point 3", "taskDescription": "Separates the third time point MP3 file into two separate MP3 files - one containing only the instrumental track and the other containing only the vocal track.", "inputDescription": "The MP3 URL of the third time point extracted from the YouTube video by AudioExtractorAgent.", "inputRequired": ["time-point-3-mp3-url"], "outputDescription": "Two MP3 URLs - one for the instrumental track and another for the vocal track of the third time point.", "outputName": ["time-point-3-instrumental-mp3-url", "time-point-3-vocal-mp3-url"], "promptInstruction": "" } ]

subagent1-json-array-mermaid

<!DOCTYPE html> <html> <head> <title>Mermaid Diagram Example</title> <script src="https://cdnjs.cloudflare.com/ajax/libs/mermaid/9.4.3/mermaid.min.js"></script> </head> <body> <div class="mermaid"> graph TD A[Time Point 1 MP3 URL] --> B{Extract Instrumental & Vocal From MP3 - Time Point 1} B --> C[Time Point 1 Instrumental MP3 URL] B --> D[Time Point 1 Vocal MP3 URL] E[Time Point 2 MP3 URL] --> F{Extract Instrumental & Vocal From MP3 - Time Point 2} F --> G[Time Point 2 Instrumental MP3 URL] F --> H[Time Point 2 Vocal MP3 URL] I[Time Point 3 MP3 URL] --> J{Extract Instrumental & Vocal From MP3 - Time Point 3} J --> K[Time Point 3 Instrumental MP3 URL] J --> L[Time Point 3 Vocal MP3 URL] </div> <script> mermaid.initialize({ startOnLoad: true }); </script> </body> </html>