Sample pack from YT video - Refine SubAgent & Output

Describe

Describe your task

Refine

Refine the plan

SubAgents

Review all agents

Deploy

Deploy your agent

Here is the updated SUBAGENT OVERVIEW with the SUBAGENT FINAL OUTPUT added for each subagent:

A) FULL TASK SUMMARY: Create a workflow to extract and process audio samples from a YouTube video and deliver a curated set of samples in a zip file.

B) FINAL TASK OUTPUT: A zip file containing 48 high-quality audio samples in .wav format, each lasting 1, 2, 4, or 8 beats, extracted from instrumentals and vocals at three specific timestamps of a YouTube video.

C) COMPONENTS OVERVIEW:
1. YouTube video input and audio extraction
2. Timestamp selection and stem separation
3. Tempo and beat detection
4. Sample chopping
5. Sample filtering and ranking
6. File format conversion
7. Sample compilation and zipping
8. Server upload

D) OVERVIEW OF SUBAGENTS:
1. YouTube Downloader
2. Audio Timestamp Processor
3. Stem Separator
4. Tempo Analyzer
5. Sample Chopper
6. Sample Curator
7. File Converter
8. Zip Creator
9. Server Uploader

E) SUBAGENT 1: YouTube Downloader
- Skill: #225 - Download Entire YouTube Video As MP3
DESCRIPTION: Inputs a YouTube video URL, and returns an MP3 file
INPUT NEEDED: YouTube URL
OUTPUT: MP3 URL
SUBAGENT FINAL OUTPUT: [youtube-mp3-file]

F) SUBAGENT 2: Audio Timestamp Processor
- Skill: #224 - Extract & Download Section Of YouTube Video URL
DESCRIPTION: Inputs a YouTube video URL, along with timing to extract, and returns an MP4 URL of the YouTube snippet.
INPUT NEEDED: YouTube URL, timing (secs) to cut it and duration (secs)
OUTPUT: MP4 URLs of the extracted sections
SUBAGENT FINAL OUTPUT: [youtube-timestamp-snippets]

G) SUBAGENT 3: Stem Separator
- Skill: #204 - Extract Instrumental & Vocal From MP3
DESCRIPTION: This skill takes an MP3 URL (i.e., a song) and extracts instrumental and vocal stems
INPUT NEEDED: MP3 URL
OUTPUT: MP3s of different stems, including vocal and instrumental (as both internal and external URL)
SUBAGENT FINAL OUTPUT: [audio-stems]

H) SUBAGENT 4: Tempo Analyzer
- Skill: #195 - Extract Beatpoints & Tempo of MP3
DESCRIPTION: This skill will extract beatpoints, downbeats, and tempo (BPM) from any MP3 file.
INPUT NEEDED: MP3 URL
OUTPUT: A clean breakdown of beatpoints and downbeats for the sample
SUBAGENT FINAL OUTPUT: [tempo-beatpoints-data]

I) SUBAGENT 5: Sample Chopper
- Skill: #197 - Cut Wav/mp3 Audio into Multiple Pieces/Samples
DESCRIPTION: This skill takes a wav or mp3 file, along with start/end time of audio to be extracted and returns the extracted files as a list of new wav URLs
INPUT NEEDED: MP3 URL and timings to cut (start/end time per required file to be cut)
OUTPUT: A list of all URLs of wav files (based on volume from waveform, intro, close, etc., mix of 2/4/8 beat samples)
SUBAGENT FINAL OUTPUT: [chopped-audio-samples]

J) SUBAGENT 6: Sample Curator
- REQUIRED: A skill that analyzes and ranks the chopped samples based on metrics like average amplitude using a library like Librosa, and selects the top 48 samples.
INPUT NEEDED: WAV sample file URLs
OUTPUT: Ranked list of the top 48 sample filenames
SUBAGENT FINAL OUTPUT: [top-48-samples]

K) SUBAGENT 7: File Converter
- Skill: #233 - Create WAV From MIDI File (Repurposed to convert MP3 to WAV)
DESCRIPTION: This subagent will convert the selected top 48 MP3 sample files to WAV format for final output.
INPUT NEEDED: MP3 sample file URLs
OUTPUT: WAV file URLs of the converted audio samples
SUBAGENT FINAL OUTPUT: [wav-samples]

L) SUBAGENT 8: Zip Creator
- Skill: #175 - Zip Internal URLs
DESCRIPTION: Input between 1 and 100 file URLs that you want to zip up. The output is the URL of the zip file.
INPUT NEEDED: Internal URLs of the 48 WAV audio files
OUTPUT: URL of the zip file containing the curated audio samples
SUBAGENT FINAL OUTPUT: [sample-zip-file]

M) SUBAGENT 9: Server Uploader
- Skill: #168 - Save text/html to server (Repurposed to save the zip file to the server)
DESCRIPTION: This will save the zip file to the server. The output of any given task is automatically saved for the duration of a particular objective being executed.
INPUT NEEDED: Zip file
OUTPUT: URL or path to access the uploaded zip file on the server
SUBAGENT FINAL OUTPUT: [server-zip-url]

Final Exact Output You Want ⚠️IMPORTANT⚠️

Expand

Other Possible SubAgents (Consider Adding These Above) ☝️🤔

Expand

#178 - Rewrite Text Based On Instructions
DESCRIPTION: This skill uses a LLM (typically GPT4) to return some type of text, based on prompt instructions. This is a fundamental and important skill.
INPUT NEEDED: 
OUTPUT: Outputted text (or save EVERYTHING AS LINK and then import everything from link? May make it easier, and only adds a few seconds to each call? but just means like 100x the files being on server! hmmm)

#184 - Brainstorm Related Keywords
DESCRIPTION: This skill inputs a single keyword (or topic, product description, or any piece of text really) and will then run multiple searches (using keyword tools like SEMRush along with Google related suggestions, etc), and then return a list of the 4 most related keywords in order of relevancy, along with additional commentary on each (as written by a LLM). 
INPUT NEEDED: single keyword (or topic? or product description, or anything?)
OUTPUT: list of 4 related keywords, listed in order, with comments

#212 - Find/Replace Bulk Files
DESCRIPTION: This skill finds/replaces text within 1-10 files stored internally on the server
INPUT NEEDED: saved file URLs + what to find/replace
OUTPUT: confirmation that find/replace has happened and new URL

#213 - Get Images with Brave API
DESCRIPTION: This skill inputs a keyword (and optional use case, aspect ratio of other requirements for the images) and finds public images for any keyword. This is a very powerful tool which will search the entire Internet, and so will always return images. This is useful for adding images to an article or video.
INPUT NEEDED: type of images I want to get (keyword and aspect ratio)
OUTPUT: List of 5 most relevant images with internal URL and description

#234 - Search CSV URL For A Keyword
DESCRIPTION: Input a CSV URL and a keyword, and skill will search the CSV and return a summary of rows that contain this keyword
INPUT NEEDED: CSV + keyword
OUTPUT: text output of results

#236 - Extract content of a file (pull in html or text from a file)
DESCRIPTION: Simple skill that extracts content of a URL, for use in other tasks (note: most tasks do not require this as extracting content from a file is already built into the task)
INPUT NEEDED: URL of a text/html
OUTPUT:

#240 - Extract all audio stems 
DESCRIPTION: This skill will input an MP3 URL and extract all instruments and variations, then return between 1-10x MP3s. MP3s will be returned in order of volume/amplitude (ie most loudest first), and only stems with a minimum level of loudness will be returned (so if an MP3 has no woodwind instruments, then that audio will not be included). Stems that can be extracted include: Vocals, instrumental, drums, bass guitar, electric guitar, acoustic guitar, piano, synthesizer, strings, wind
INPUT: MP3 URL 
OUTPUT: 1-10x MP3 URLs (arranged in order of importance)

#215 - Summarise and Return All Final Output Files
DESCRIPTION: This skill returns a summary of various previous skills that have completed, and is particularly useful when appearing at the end of an entire Agent task, silo or sequence of skills.
INPUT NEEDED: at least one output (perhaps other supplementary ones that may be helpful)
OUTPUT: nice text summary containing 1 or more outputs

Task Summary (You Wrote)

Expand

I want to input a youtube video, then get a collection of music samples returned to me (.wav files in a zip). I think I should specify 3 specific times (eg 5 seconds, 130 seconds, 190 seconds, and then the agent  rip the audio file from YouTube, then extract stems (instrumentals, vocals), and then extract (say) 16 beats worth of audio - beginning from the 3 timings I've entered. This gives us 3x separate audio sections to sample, and if we then separate them into vocals and instrumentals that then gives us 6x separate audio files to sample from (although we will use the initial tempo/beatpoint for all 6).

Next we will want to use the beatpoint and tempo to extract samples (eg 1/2/4/8 beat in lengths, beginning from the beatpoints). So we chopup the samples eg using SOX, perhaps chopping based on beat length (if BPM = 90bpm, then 1 beat is 0.66secs) and beat position, and chopping maybe a mix of 1/2/4/8 beat samples.

This will likely give us far too many samples, so we may want to finally filter them down (to "the best 48 samples"). We could do this by using a library like Librosa to filter somehow (eg based on average amplitude), and rank the samples. We will likely need to calculate how many samples we are going to get, how/if to rank them, and then generate them acccordingly. I am not sure whether we should filter or just do a bulk import.

Finally we will want to save all the files to a server and zip them up.

Diagram Of Entire Task (All SubAgents)

Expand Diagram

Ready To Proceed?

When you are ready to proceed, click the Save button to load the next step...
The next step is SubAgents. It will take around 10 minutes to load this page...
If you are entirely unhappy with this page, click Back to rerun Step 1 again.
Before saving, check the SubAgents and Output fields above (very important!)
The Agent heavily relies on the SubAgents and Output fields to build your task.

-- The LLM has raised some questions to help you clarify the desired final `output` (important - this must be specific) 
-- These ideas are worth scanning briefly (then either amend output/subagents -- or go back to step 1)
-- The first few points regarding how to clarify the `output` are particular important!
-- If you need to make your output more specific, change it (on the previous tab)...

## QUESTIONS RAISED BY THE LLM:

Thank you for the detailed description of the task you want to automate using an AI Agent. Let me address your questions one by one:

a) Questions to clarify the final output:
- How many music samples should the final output contain?
- What format should the music samples be in? (e.g., .wav, .mp3)
- Should the samples be labeled or named in a specific way?
- Is there a specific duration range for the samples? (e.g., 1-2 seconds, 2-4 seconds)
- Should the samples be organized in a specific folder structure within the zip file?
- Are there any specific requirements for the audio quality or bitrate of the samples?

b) Inputs the task should accept before launching the task completion process:
- The YouTube video URL or ID
- The three specific timings within the video to extract samples from
- The desired number of samples to generate (if not fixed to 48)
- Optional: A naming convention for the samples, if required

c) Potential task segmentation:
I believe the task can benefit from segmentation in the following ways:
1. Audio extraction and stem separation: This involves extracting the audio from the YouTube video and separating it into instrumental and vocal stems. This could be a separate task to ensure the quality of the extracted audio and the accuracy of the stem separation.
2. Sample generation: This task focuses on generating the actual music samples based on the specified timings, beat lengths, and the extracted audio stems. It could be further divided into subtasks for each timing and stem combination.
3. Sample filtering and ranking: Once the samples are generated, a separate task could handle filtering and ranking them based on criteria like average amplitude or other audio features. This ensures that the final output contains the best quality samples.
4. Packaging and delivery: The final task would involve saving the selected samples to a server, organizing them in the desired folder structure, compressing them into a zip file, and delivering the output to the user.

By segmenting the tasks in this manner, each component can be handled separately, allowing for better control over the quality and efficiency of the overall workflow.

I hope this clarifies the final output and provides some insights into structuring the AI Agent's workflow for this task. Let me know if you have any further questions!

Expand

a) Questions to clarify the final output:

1. You mentioned that you want to get samples in different beat lengths and from different sections from the video. Do these beat lengths and sections need to be of a certain quality or characteristics? 
2. You mentioned "the best 48 samples", what criteria should we use to define "the best" - is this defined by amplitude, variance in frequency, or something else? 
3. What should be the quality of the .wav files that are returned - the bitrate, the sampling rate, etc?
4. Do you want the zip file to have a specific naming convention?
5. Is there a specific file size limit for the zip file, and what should be done if the size limit is exceeded?
6. Is there a specific server or platform where the zipped files should be uploaded?

b) Inputs the task should accept:

1. The YouTube video URL from which the audio needs to be ripped.
2. The specification of the 3 specific times in seconds from the video to extract the audio samples from.
3. Criteria for selecting "the best" samples - for instance, does the amplitude need to exceed a certain threshold?
4. Desired beat lengths for chopping the samples (1/2/4/8).
5. Tempo of the song in Beats Per Minute (BPM), if needed for chopping the samples.
6. Access details of the server where the final zip file needs to be uploaded, if any.
7. Any specific preference in naming conventions of the files and folders in the resultant zipped collection.

Research Links

Splice Sample Pack Vocal Chops Instrumental Loops Curated Samples

GitHub Repository Creating Sample Packs from Audio Video Files

Blog Post Tutorial Music Producer Creating High-Quality Sample Packs

Ableton Live FL Studio Using Sample Packs Screenshots Demo Videos

Sample Pack Creation Tool Service Output Specifications User Interface

☝️ Like These? Consider Bringing These Ideas Into Output Or SubAgents (on previous page)

-- The LLM has suggested possible templates that could be useful to build this agent. 
-- Consider preparing these now...(you can upload them as `pre-prompts` in the next stage).
-- These prompts must be text, HTML or a URL (eg of an image or some folder).

## TEMPLATE IDEAS FROM THE LLM:

Based on the description of the final output and the capabilities of the AI agent, here are some suggestions for templates and resources that could help refine the definition of a successful output:

1. Product pages of sample packs on music production marketplaces:
   - Visit websites like Splice, Loopmasters, or Beatport and search for sample packs similar to the desired output (e.g., "vocal chops," "instrumental loops," "curated samples").
   - Read the product descriptions, which often include details about the number of samples, file format, length, and quality.
   - Look at customer reviews to gauge what users find valuable in a sample pack.

2. README files of open-source sample pack projects on GitHub:
   - Search for repositories on GitHub related to creating sample packs from audio or video files.
   - Review the README files, which may contain descriptions of the project's goals, outputs, and methodologies.
   - Analyze the code structure and libraries used to gain insights into the technical implementation.

3. Blog posts or tutorials on creating sample packs:
   - Search for blog posts, articles, or tutorials written by music producers or audio engineers on creating sample packs.
   - Pay attention to their recommendations on sample lengths, file formats, naming conventions, and curation processes.
   - Note any specific techniques or tools they use to ensure high-quality output.

4. Screenshots or demo videos of music production software using sample packs:
   - Look for screenshots or demo videos showcasing how sample packs are typically used within music production software like Ableton Live, FL Studio, or Logic Pro.
   - Observe how the samples are organized, labeled, and accessed within the software to inform your own output structure.

5. Descriptions or previews of sample pack creation tools or services:
   - Research existing tools or services that automate the process of creating sample packs from audio or video files.
   - Analyze their feature sets, output specifications, and user interfaces to identify best practices and potential improvements for your own workflow.

By collecting and reviewing these templates, you can gain valuable insights into what constitutes a high-quality, user-friendly, and professionally curated sample pack. This information can then be used to refine your definition of a successful output and guide the development of your automated workflow.

Final Points Before Saving!

If you are at all unsure about this step, consider returning to step 1 (having taken new ideas from this Step 2 page)
DO NOT proceed to Step 3 if the information on this page has raised fundamental problems with your initial task
If the original description you wrote on Step 1 is largely correct, and this page has not really changed your mind about anything, then simply tweak the Output and SubAgents fields and Save (you can always hit BACK later)
IMPORTANT: If your initial description (Step 1) and Subagent & Output (Step 2) are flawless then the Agent will take it from here and your task will be easy to build. BUT if you are already seeing problems, consider hitting BACK and starting again.
It is MUCH easier to go back a step than to try to refine and imperfect task down the line. Final chance to start again!
Remember - the initial description on Step 1, along with the SubAgent summary & Output spec on Step 2, are the foundation on which all remaining steps and Agents run. This step is here for a reason, so please consider hitting back. It will save time later!

Need To Start Afresh?

BACK TO DESCRIBE

Sample pack from YT video - Refine SubAgent & Output

Describe

Refine

SubAgents

Deploy

Final Exact Output You Want ⚠️IMPORTANT⚠️

Expand

Other Possible SubAgents (Consider Adding These Above) ☝️🤔

Expand

Task Summary (You Wrote)

Expand

Diagram Of Entire Task (All SubAgents)

Expand Diagram

Ready To Proceed?

More Questions & Research Ideas (LLM #2)

Expand

Research Links

☝️ Like These? Consider Bringing These Ideas Into Output Or SubAgents (on previous page)

Final Points Before Saving!

Need To Start Afresh?

Tweaked & Good To Go?