The Bioinformatics Core has written open-source bioinformatics software for the entire community to use and is freely available on our GitHub page. We have also created an AMI (Amazon Machine Image) that runs Galaxy loaded with standard as well as more customized software. You can use this AMI for command-line analysis as well. Our current public AMI ID is ami-f4733f94. See below for more detailed instructions on how to launch instances using our AMI.
A windowed adaptive trimming tool for FASTQ files using quality: download from GitHub
Most modern sequencing technologies produce reads that have deteriorating quality towards the 3′-end and some towards the 5′-end as well. Incorrectly called bases in both regions negatively impact assembles, mapping, and downstream bioinformatics analyses.
Sickle is a tool that uses sliding windows along with quality and length thresholds to determine when quality is sufficiently low to trim the 3′-end of reads and also determines when the quality is sufficiently high enough to trim the 5′-end of reads. It will also discard reads based upon the length threshold. It takes the quality values and slides a window across them whose length is 0.1 times the length of the read. If this length is less than 1, then the window is set to be equal to the length of the read. Otherwise, the window slides along the quality values until the average quality in the window rises above the threshold, at which point the algorithm determines where within the window the rise occurs and cuts the read and quality there for the 5′-end cut. Then when the average quality in the window drops below the threshold, the algorithm determines where in the window the drop occurs and cuts both the read and quality strings there for the 3′-end cut. However, if the length of the remaining sequence is less than the minimum length threshold, then the read is discarded entirely.
A very simple adapter trimmer: download from GitHub
Scythe uses a Naive Bayesian approach to classify contaminant substrings in sequence reads. It considers quality information, which can make it robust in picking out 3′-end adapters, which often include poor quality bases.
The Bayesian approach Scythe uses compares two likelihood models: the probability of seeing the matches in a sequence given contamination, and not given contamination. Given that the read is contaminated, the probability of seeing a certain number of matches and mismatches is a function of the quality of the sequence. Given the read is not contaminated (and is thus assumed to be random sequence), the probability of seeing a certain number of matches and mismatches is chance. The posterior is calculated across both these likelihood models, and the class (contaminated or not contaminated) with the maximum posterior probability is the class selected.
Quick Read Quality Control: download from GitHub
qrqc is a fast and extensible R package that reports basic quality and summary statistics on FASTQ and FASTA files, including base and quality distribution by position, sequence length distribution, and common sequences.
A barcode demultiplexing and trimming tool for FASTQ files: download from GitHub
Next-generation sequencing can currently produce hundreds of millions of reads per lane of sample and that number increases at a dizzying rate. Barcoding individual sequences for multiple lines or multiple species is a cost-efficient method to sequence and analyze a broad range of data.
Sabre is a tool that will demultiplex barcoded reads into separate files. It will work on both single-end and paired-end data in fastq format. It simply compares the provided barcodes with each read and separates the read into its appropriate barcode file, after stripping the barcode from the read (and also stripping the quality values of the barcode bases). If a read does not have a recognized barcode, then it is put into a separate “unknown” file. Sabre also has an option to allow mismatches of the barcodes.
Bioinformatics Core Galaxy and Command-Line AMI
The Bioinformatics Core uses Galaxy and the command-line for our training workshops and courses, running in the Amazon Cloud. We make the Amazon Machine Image (AMI) publicly available so that the community can use it for their projects. In addition to the standard software, our AMI contains customized software and interfaces that you will not find elsewhere; these tools are available through the Galaxy interface, or via the command-line using modules (or directly, under /software). The AMI also contains all of the training materials from our week-long workshops as well as pre-indexed model genomes in Galaxy and under /data/refs. The current Bioinformatics Core AMI ID is ami-f4733f94 and is located in the N. California Region. There is no charge for using this AMI to launch your own instances in the Amazon Cloud, but you will need an AWS account, and Amazon will charge you for running instances and storing/transferring data. The default Galaxy admin login on this AMI is email@example.com and the password is galaxy. To log in on the command-line, use your ssh key with the login/username ubuntu.
Notes on starting our AMI: If you are using Galaxy, you need to use the following rules in your security group:
- Custom TCP Rule, port range 8080
- Custom TCP Rule, port range 2200
- Custom TCP Rule, port range 20-21
Increasing the disk storage on an instance
You can increase the size of your data partition (Device /dev/sdg) on launch by simply increasing the size of the data volume from 200Gb to whatever you want (up to 16384 Gb) in the “Add Storage” step. There are also instructions on how to increase the data volume size of an existing instance when you want to increase your capacity. We have added code in our AMI to automatically detect if the volume has increased capacity and to expand the filesystem to that capacity. The steps to increase to maximum capacity can take a while depending upon the size increase, which means you may need to wait a while before the instance is up and running.
Using FTP and DataManagers for Galaxy
If you wish to use FTP to transfer files to your Galaxy instance, we have instructions on how to use FileZilla to do so. FTP transfer is recommended for large files, however, you can use whatever FTP client you want. We also have instructions on how to use DataManagers to create your own built-in/locally cached indexed genomes in Galaxy.
Copying AMIs to different Regions
To copy our AMI to a different Region, you’ll need to launch an instance using our AMI and then make your own AMI from that instance. Then follow these instructions to copy the AMI to another Region. Make sure to terminate the instance, deregister the AMI, and delete the snapshots of your AMI from the N. California region.