SLiMSuite 2014-04-22 now available

A new download of SLiMSuite (release 2014-04-22) is now available. As well as fixing the gopher.py error, the download page and readme have had a slight makeover, which should make them load quicker.

As part of ongoing consolidation and documentation, SeqSuite has now been incorporated into in a single SLiMSuite download. (Previously, SLiMSuite was available as a reduced set of programs and SeqSuite had the full set.) The intention is to retire the SeqSuite moniker over the coming months, although the programs themselves will still be available.

The lastest release also features a new program, SLiMFarmer, for running (Q)SLiMFinder and SLiMProb batch jobs on parallel processors. SLiMFarmer is still under development and should hopefully work with other SLiMSuite programs too but has not yet been tested.

Other miscellaneous updates are listed below.

Updates since last release:

• comparimotif_V3: Updated from Version 3.10.
→ Version 3.10: Added forking.
→ Version 3.11: Added additional overlap/matchfix checks during basic comparison to try and speed up.
→ Version 3.12: Replaced deprecated sets.Set() with set().

• gablam: Updated from Version 2.11.
→ Version 2.12: Consolidated use of BLAST V2.

• haqesac: Updated from Version 1.9.
→ Version 1.10: Added exceptions for BLAST failure.

• picsi: Updated from Version 1.1.
→ Version 1.2: Updated to BUDAPEST 2.3 and rje_mascot.

• pingu_V4: Created.
→ Version 4.0: Initial Compilation based on code from SLiMBench and PINGU 3.9 (inherited as pingu_V3).
→ Version 4.1: Adding compilation of PPI databases using new rje_xref V1.1 and older objects from PINGU V3.
→ Version 4.2: Bug fixes for use of PPISource to create PPI databases.

• qslimfinder: Updated from Version 1.6.
→ Version 1.7: Fixed "MustHave=LIST" correction of motif space.

• seqmapper: Updated from Version 2.0.
→ Version 2.1: Added catching of failure to read input sequences. Removed 'Run' from GABLAM table.

• slimbench: Updated from Version 2.0.
→ Version 2.1: Fixed memsaver=T unless in development mode (dev=T). Removed old Assessment. Tested with simbench analysis.
→ Version 2.2: Replaced searchini=LIST with searchini=FILE and moved to SimBench commands.
→ Version 2.2: Modified the FN/TN and ResNum calculations. No longer rate TP in random data as OT.

• slimfarmer: Created.
→ Version 0.0: Initial Compilation.
→ Version 1.0: Functional version using rje_qsub and rje_iridis to fork out SLiMSuite runs.
→ Version 1.1: Updated to use rje_hpc.JobFarmer and incorporate main SLiMSuite farming within SLiMFarmer class.

• slimfinder: Updated from Version 4.5.
→ Version 4.6: Minor modification to seqocc=T function. !Experimental! Added main occurrence output and modified savespace.

• slimmutant: Created.
→ Version 0.0: Initial Compilation.
→ Version 1.0: Working version with standalone functionality.

• slimprob: Updated from Version 1.0.
→ Version 1.1: Tidied import commands.
→ Version 1.2: Increased extras=X levels. Adjusted maxsize=X assessment to be post-masking.

• ned_rankbydistribution: Updated from Version 1.1.
→ Version 1.2: Replaced depracated Set module.

• rje: Updated from Version 4.8.
→ Version 4.9: Added rje.slimsuite, which determines the slimsuite home directory from rje.py file path.
→ Version 4.10: Added osx=T/F option for Mac-specific running options.

• rje_blast_V2: Updated from Version 2.4.
→ Version 2.5: Minor modifications for SLiMCore UPC generation.
→ Version 2.6: Minor bug fixes.

• rje_db: Updated from Version 1.2.
→ Version 1.3: Minor modifications for SLiMCore FUPC development.
→ Version 1.4: Added list checking with addEmptyTable.

• rje_dismatrix_V2: Updated from Version 2.9.
→ Version 2.10: Minor modifications for SLiMCore UPC.

• rje_genemap: Updated from Version 1.4.
→ Version 1.5: Minor tweak of expected HGNC input following change to downloads.

• rje_hpc: Created.
→ Version 1.0: Initial Compilation based on rje_iridis V1.10.

• rje_iridis: Updated from Version 1.9.
→ Version 1.10: Modified freemem setting to run on Katana. Made rsh optional. Removed defunct IRIDIS3 option.

• rje_obj: Updated from Version 1.3.
→ Version 1.4: Added sourceDataFile() method from SLiMBench for wider use.
→ Version 1.5: Added 'basestr' and 'basefile' cmdlist types.
→ Version 1.6: Added osx=T/F option for Mac-specific running options.

• rje_qsub: Updated from Version 1.4.
→ Version 1.5: Added emailing of job stats after run. Added vmem limit.

• rje_seq: Updated from Version 3.17.
→ Version 3.18: Minor BLAST+ bug fixes. Added exceptions to readBLAST failure.

• rje_seqlist: Updated from Version 1.3.
→ Version 1.4: Added dna2prot reformat function.

• rje_slimcore: Updated from Version 1.12.
→ Version 1.13: Modified the savespace settings to reduce numbers of files. targz file now uses RunID not Build Info.
→ Version 1.14: Started adding code for Fragmented UPC (FUPC) clustering.

• rje_slimlist: Updated from Version 1.2.
→ Version 1.3: Added auto-download of ELM data.

• rje_uniprot: Updated from Version 3.14.
→ Version 3.14: Added dblist=LIST and dbsplit=T/F for additional DB link output control. Set unipath default to url.
→ Version 3.15: Added extraction of taxonomic groups. Add UniFormat to improve pure downloads.
→ Version 3.16: Added WBGene ID's from WormBase as one of the recognised DB XRef to parse.
→ Version 3.17: Efficiency tweak to URL-based extraction of acclist.
→ Version 3.18: Minor modification to database parsing.

• rje_xref: Updated from Version 1.0.
→ Version 1.1: Added output of ID lists to text files. Major reworking. Tested with HPRD and HGNC.

Missing gopher.py file

There is a bug with the current software download, with a file missing from the libraries/ directory. The download will hopefully be updated soon but in the meantime please email richard.edwards[at]unsw.ed.au and I will send you the file.

New downloads and fixed webpages

New releases of SeqSuite and SLiMSuite are now available. The webpages have now hopefully been fixed too, including the broken Manual links. (A bit of trouble parsing some the docstrings had messed up the HTML, in case you care!) Please report any more anomalies.

There are not many major updates since the last release. The biggest are that SLiMFinder (and QSLiMFinder) now produce a single *.occ.csv containing motif instances for all datasets, in addition to the old dataset-specific files. This is to make the output more consistent with SLiMProb although do note that some of the column headers are different. The new file contains the same data as the old dataset-specific *.occ.csv files plus two additional columns: Dataset and RunID. (These match the main *.csv output.)

Dataset-specific results files have also been cleaned up a little for (Q)SLiMFinder and SLiMProb (i.e. the SLiMCore Class in libraries/rje_slimcore) to make the targz=T/F and savespace=X options a little more useful and consistent. This will be the subject of another post shortly.

Other miscellaneous updates are listed below.

Updates since last release:

• comparimotif_V3: Updated from Version 3.10.
→ Version 3.10: Added forking.
→ Version 3.11: Added additional overlap/matchfix checks during basic comparison to try and speed up.

• qslimfinder: Updated from Version 1.6.
→ Version 1.7: Fixed "MustHave=LIST" correction of motif space.

• slimfinder: Updated from Version 4.5.
→ Version 4.6: Minor modification to seqocc=T function. !Experimental! Added main occurrence output and modified savespace.

• rje_pydocs: Updated from Version 2.8.
→ Version 2.8: Added docsource=PATH : Input path for Python Module documentation (manuals etc.) ['../docs/']
→ Version 2.9: Attempts to fix some broken links and sort out manuals confusion

• rje_slimcore: Updated from Version 1.12.
→ Version 1.13: Modified the savespace settings to reduce numbers of files. targz file now uses RunID not Build Info.

• rje_uniprot: Updated from Version 3.14.
→ Version 3.14: Added dblist=LIST and dbsplit=T/F for additional DB link output control. Set unipath default to url.
→ Version 3.15: Added extraction of taxonomic groups. Add UniFormat to improve pure downloads.

New Software Release

New releases of SLiMSuite and SeqSuite are now available. Please note that RJESuite has now been discontinued - for simplicity, all of the extra gubbins is now part of the SeqSuite release. SLiMSuite still represents a cut-down version that focuses on Short Linear Motif analysis tools.

There have been a number of updates since the last release, which will be the focus of future posts. The biggest change since the last release is the implementation of BLAST+ as the default in place of BLAST for most tools. The old BLAST can still be invoked using the oldblast=T switch. In addition to blastpath=PATH, a new blast+path=PATH parameter will need to be set.

Apart from some file organisation tweaks, the other major change is that CompariMotif now has a memsaver=T mode, which will process very large motif lists much quicker and avoid memory issues. The XGMML output is not (yet) available in this mode. For multi-processor CPUs and large searchdb motif lists, CompariMotif now also supported forking (forks=X).

Documentation is in the process of having an overhaul and is still lagging behind as a result. Please ask if anything is unclear and that section of documentation will be prioritised.

Updates since last release:

• aphid: Updated from Version 2.0.
→ Version 2.1: Reduced import commands.

• budapest: Updated from Version 2.1.
→ Version 2.2: Removed unrequired rje_dismatrix import.
→ Version 2.3: Updated to use rje_blast_V2. Needs further updates for BLAST+. Deleted obsolete OLDreadMascot() method.

• comparimotif_V3: Updated from Version 3.9.
→ Version 3.10: Added MemSaver option, which will read and process input motifs (not searchdb) one motif at a time.
→ Version 3.10: Added forking.

• fiesta: Updated from Version 1.5.
→ Version 1.6: Removed HAQESAC import (uses MultiHAQ).
→ Version 1.7: Updated to use rje_blast_V2. Needs work to make function with BLAST+.

• gablam: Updated from Version 2.10.
→ Version 2.11: Altered to use BLAST+ and rje_blast_V2.

• gasp: Updated from Version 1.3.
→ Version 1.4: Minor tweaks to imports.

• gfessa: Updated from Version 1.2.
→ Version 1.3: Tidied module imports.
→ Version 1.4: Switched to rje_blast_V2. More work needed for BLAST+.

• haqesac: Updated from Version 1.8.
→ Version 1.9: Added rje_blast_V2 implementation and BLAST+. Use oldblast=T for old BLAST.

• peptcluster: Updated from Version 1.3.
→ Version 1.4: Bug fixes for end of sequence characters and different length peptides.

• picsi: Updated from Version 1.0.
→ Version 1.1: Updated to blast_V2 and BLAST+.

• pingu: Updated from Version 3.8.
→ Version 3.9: Tidied imports.

• qslimfinder: Updated from Version 1.5.
→ Version 1.6: Removed excess module imports.

• slimbench: Updated from Version 1.8.
→ Version 1.9: Added memsaver option. Replaced SLiMSearch with SLiMProb. Altered default IO paths.
→ Version 1.9: Removed 3DID again: new ELM interaction_domains file has position-specific PPI details.
→ Version 2.0: Major overhaul of input options to standardise/clarify. Implemented auto-downloads and PPI datasets.

• slimprob: Updated from Version 1.0.
→ Version 1.1: Tidied import commands.

• slimsuite: Created.
→ Version 0.0: Initial Compilation with downloadelm function.

• rje_pydocs: Updated from Version 2.6.
→ Version 2.7: Added rje_ppi output for module links.
→ Version 2.8: Added parsing of commandline options from docstring and cmdRead calls.
→ Version 2.8: Added docsource=PATH : Input path for Python Module documentation (manuals etc.) ['../docs/']

• rje: Updated from Version 4.6.
→ Version 4.7: Added self.warn list and self.warnLog() functions to Log object. Modified i=-1 quitchoice to raise not quit.
→ Version 4.8: Added perc cmdtype = float that is multiplied by 100.0 if < 1.0. Removed server option from iniCmds().

• rje_ancseq: Updated from Version 1.2.
→ Version 1.3: Changed "biproblem" error handling in gaspProbs()

• rje_blast_V1: Updated from Version 1.14.
→ Version 1.15: Added OldBLAST/Legacy option to Object for compatibility with rje_blast_V2. (Always True!)

• rje_blast_V2: Updated from Version 2.1.
→ Version 2.2: Added gablamData() to return old-style GABLAM dictionary from table.
→ Version 2.3: Added blastCluster() method to return UPC clustering and GABLAM distance matrix from a file.
→ Version 2.4: Scrapped BLAST "Run" field to simplify code - keep a single run per BLASTRun object.

• rje_db: Updated from Version 1.0.
→ Version 1.1: Added sortedEntries() function.
→ Version 1.2: Added Table.hasField(field). Add openTable(), readEntry() and readSet() methods.

• rje_forker: Created.
→ Version 0.0: Initial Compilation.

• rje_iridis: Updated from Version 1.8.
→ Version 1.9: Added scanning of legacy folder - moving GOPHER_V2!

• rje_obj: Updated from Version 1.0.
→ Version 1.1: Added rje_zen import and self.zen() to call rje_zen.Zen().wisdom().
→ Version 1.2: Added warnLog functions.
→ Version 1.3: Added perc cmdtype = float that is multiplied by 100.0 if < 1.0. Also added cmdtype = date for YYYY-MM-DD.

• rje_ppi: Updated from Version 2.7.
→ Version 2.8: Tweaked Spring Layout. Stores original Hub and Spoke Field.

• rje_seq: Updated from Version 3.16.
→ Version 3.17: Updated to use BLAST+ and rje_blast_V2

• rje_sequence: Updated from Version 2.2.
→ Version 2.3: Added alternative self.info keys for sequence (for UniProt splice variants). Added SpliceVar dict.

• rje_slimcore: Updated from Version 1.10.
→ Version 1.11: Tidied some of the module imports.
→ Version 1.12: Upgraded BLAST to BLAST+. Can use old BLAST with oldblast=T.

• rje_slimlist: Updated from Version 1.1.
→ Version 1.2: Added some extra functions for CompariMotif Memsaver mode

• rje_tree: Updated from Version 2.9.
→ Version 2.10: Added cleanup of *.r.csv file following R-based PNG generation.

• rje_uniprot: Updated from Version 3.13.
→ Version 3.14: Added direct retrieval of UniProt entries from URL, including full proteomes. Updated output file naming.
→ Version 3.14: Added dblist=LIST and dbsplit=T/F for additional DB link output control. Set unipath default to url.

• rje_xml: Updated from Version 0.1.
→ Version 0.2: Added parsing from URL.

• rje_xref: Updated from Version 0.0.
→ Version 1.0: Added xfrom and xto fields and xMap() function for mapping from one ID set to another.

Updated programs coming soon...

SLiMSuite and Seqsuite have been undergoing some tidying and additional tweaks, such as implementing BLAST+ in most programs. The documentation is also undergoing a bit of an overhaul (see the Documentation links in the left sidebar) and so the distribution of the latest code is being held back for a while. If you want access to the latest versions, however, feel free to get in touch. (Particularly if you want to use BLAST+ with SLiMSuite or HAQESAC.)

Availability, Installation and Setup

SLiMSuite and Seqsuite are currently available from http://bioware.soton.ac.uk as three packages:

1. SLiMSuite contains software for Short Linear Motif (SLiM) analysis.
2. SeqSuite contains all of the SLiMSuite programs plus some additional sequence analysis programs.
3. RJESuite contains SLiMSuite, SeqSuite and a bunch of other miscellaneous utilities and bits and bobs.

In future, it is envisaged that a single Git repository will contain all the relevant code and documentation.

All three packages have the same basic installation, directory structure and setup requirements. For basic functionality, no other setup should be necessary beyond downloading and unzipping the package in the desired directory if Python is installed on your system. Some programs will need to use external components or accessory applications, which may need additional installation.

If you do not have Python, you can download it free from www.python.org at http://www.python.org/download/. The modules are written in Python 2.x and most have been tested with 2.7. The Python website has good information about how to download and install Python but if you have any problems, please get in touch and I will help if I can.

All the required files should have been provided in the download zip file. The Python Modules are open source and may be changed if desired, although please give me credit for any useful bits you pillage. I cannot accept any responsibility if you make changes and the program stops working, however! If you want some help understanding the way the modules and classes are set up so you can edit them, just contact me.

Directory Structure

Once unzipped, the download will unpack a top level seqsuite/ or slimsuite/ directory with the following subdirectories:

data/ contains example data for testing programs. (Currently under development.)

docs/ contains documentation.

extras/ contains accessory programs that are not part of the main program suite.

legacy/ contains superseded programs that are no longer supported. (Currently under development.)

libraries/ contains all the python libraries used by the main tools (and extras), some of which have standalone functionality.

settings/ contains INI files set default options.

tools/ contains the main program suite.

NOTE: It is recommended that analyses are performed outside these directories for ease of reinstallation.

Third party software

Many of the tools make use of third party software. Where possible, instructions will be provided for obtaining these programs but a quick Google is usually sufficient - wherever possible, third party software is free for academic use and (ideally) open source.

When third party software is used, SeqSuite will also need to the path to the program, or suite of programs. This will be covered more in the Command-line Options section but BLAST and clustalw deserve a special mention as examples because many of the programs use these as default programs for certain functions.

BLAST is actually a suite of programs and the path containing these executables should be provided using `blastpath=PATH/', e.g.:

blastpath=/usr/ncbi/bin/

For BLAST, do not give the full path to the program (e.g. blastpath=/usr/ncbi/bin/blastp). BLAST cannot be replaced easily by other programs. BLAST has now largely been superseded by BLAST+, which needs its own path parameter:

blast+path=PATH

Some programs are still restricted to BLAST at the moment and other tools can be made to use the BLAST with the oldblast=T switch.

Clustalw is a useful standalone program that is used as a default for alignments and trees in the absence of newer (better) programs. For this, and other single executables, the full path to the program is given:

clustalw=/usr/bioware/clustalw1.83/clustalw

In these situations, a different program with the same input and output can be substituted.

NOTE: Remember to set the relevant paths in an appropriate *.ini file in settings/. Where possible, error messages will identify issues with third party software but due to a lack of testing on a diversity of systems, this is not always possible. If a program crashes, please check the *.log file for signs that there may be a problem with the installation and/or path given for third party programs, such as BLAST.

Upgrading

At present, each upgrade is distributed as a separate package. You can check the current version by the date in the name of the distribution file (in ISO 8601 standard, YYYY-MM-DD format). Plans are afoot to switch to a Git repository, which will make upgrades easier.

New Software Release

New releases of SeqSuite, SLiMSuite and RJESuite are now available.

The biggest change since the last release is the renaming of SLiMSearch to SLiMProb. This is to avoid confusion between the old SLiMSearch 1.x (now SLiMProb) and the newer SLiMSearch 2.x webserver, which has a different range of functions.

Updates since last release:

• cpppred: Created.

• gopher: Updated from Version 3.1.
→ Version 3.2: Minor tweak to prevent unwanted directory generation for programs using existing GOPHER alignments.
→ Version 3.3: Added rje_blast_V2 to use BLAST+. Run with legacy=T to stick with old NCBI BLAST. Started utilising rje_seqlist.

• pepbindpred: Created.

• slimprob: Created.
→ Version 1.0: SLiMProb 1.0 based on SLiMSearch 1.7. Altered output files to be *.csv and *.occ.csv.

• file_monster: Updated from Version 2.0.
→ Version 2.1: Added dirsum function.

• rje: Updated from Version 4.5.
→ Version 4.6: Added dev and warn options.

• rje_blast_V2: Created.
→ Version 2.0: Initial Compilation from rje_blast_V1 V1.14.
→ Version 2.1: Tweaking code to work with GOPHER 3.x - removing self.info etc. Added blastObj() method.

• rje_db: Updated from Version 0.4.
→ Version 0.5: Initial coding of index mode. (Not yet fully functional.)
→ Version 1.0: Working, so upgraded to version 1.0!

• rje_obj: Updated from Version 0.0.
→ Version 1.0: Fully working version, so upgraded to 1.0. Added dev and warn options.

• rje_seq: Updated from Version 3.15.
→ Version 3.16: Added BLAST+ path and seqFromBlastDBCmd()

• rje_slimcalc: Updated from Version 0.5.
→ Version 0.6: Minor tweak to avoid unwanted GOPHER directory generation.
→ Version 0.7: Added RLC to "All" conscore running.

• rje_slimcore: Updated from Version 1.9.
→ Version 1.10: Bypass UPC generation for single sequences.

Documentation is still in the process of development. BLAST+ implementation is ongoing - please get in touch if this is something you need.

New Software Release

New releases of SeqSuite, SLiMSuite and RJESuite are now available.

Updates since last release:

• budapest: Updated from Version 2.0.
→ Version 2.1: Improved handling of iTRAQ data using rje_mascot V1.2.

• comparimotif_V3: Updated from Version 3.8.
→ Version 3.8: Changed scoring of overlapping ambiguities - uses IC of all possible ambiguities. Added "Ugly" match type.
→ Version 3.9: Added xgformat=T/F : Whether to use default CompariMotif formatting or leave blank for e.g. Cytoscape [True]

• happi: Updated from Version 1.1.
→ Version 1.2: Added addclass and refined output for Host-Pathogen PPI analysis.

• pingu: Updated from Version 3.7.
→ Version 3.8: Hopefully fixed issue of Fasta file generation log output writing to wrong log file.

• qslimfinder: Updated from Version 1.4.
→ Version 1.4: Added qexact=T/F option for calculating Exact Query motif space (True) or estimating from dimers (False).
→ Version 1.5: Implemented SigV calculation. Modified extras setting.

• seqmapper: Updated from Version 1.2.
→ Version 2.0: Reworked with new Object format, new BLAST(+) module and new seqlist module.

• slimbench: Updated from Version 1.5.
→ Version 1.6: Added "simonly" to datatype - calculates both SN and FPR from "sim" data (ignores "ran") to check query bias.
→ Version 1.7: Added Benchmarking of ELM datasets without queries.
→ Version 1.8: Added Benchmarking dataset generation from PPI data and 3DID.

• slimfinder: Updated from Version 4.4.
→ Version 4.5: Minor modifications to fix sigV and sigPrime bugs. Modified extras setting. Added palindrome setting for DNA motifs.

• file_monster: Updated from Version 1.6.
→ Version 2.0: Major reworking with new object making use of rje_db tables etc. Old functions to be ported with time.

• rje_dbase: Updated from Version 2.2.
→ Version 2.3: Added construction of EnsEMBL TaxaDB sets during TaxaDB construction.

• rje_seqgen: Updated from Version 1.6.
→ Version 1.7: Modified/fixed ESTgen function to work for protein sequences.

• ned_rankbydistribution: Updated from Version 1.0.

• rje: Updated from Version 4.4.
→ Version 4.5: Modified randomString() and added stringShuffle() methods.

• rje_blast_V1: Created.
→ Version 0.0: Initial Working Compilation.
→ Version 0.1: No Out Object in Objects
→ Version 1.0: Corrected to work with blastn (and blastp)
→ Version 1.1: Added special calling for Cerberus
→ Version 1.2: Added GABLAM and GABLAMO to BlastHit
→ Version 1.3: Added GABLAM calculation upon reading BLAST results and clearing Alignment sequences to save memory
→ Version 1.4: Tidied up the module with improved logging and progress reporting. Added dbCleanup.
→ Version 1.5: Added checking for multiple hits with same name and modified BLAST_Run.hitToSeq()
→ Version 1.6: Added nucleotide vs protein searches to GABLAM
→ Version 1.7: Added nucleotide vs nucleotide searches to GABLAM
→ Version 1.8: Added local alignment summary output to ReadBLAST()
→ Version 1.9: Added BLAST -C
→ Version 1.10: Added BLAST -g
→ Version 1.11: Added gablamfrag=X : Length of gaps between mapped residue for fragmenting local hits [100]
→ Version 1.12: Altered checkDB and cleanupDB to spot index files split over multiple files (*.00.p* etc.)
→ Version 1.13: Added localcut=X : Cut-off length for local alignments contributing to global GABLAM stats) [0]
→ Version 1.14: Added blast.checkProg(qtype,stype) to check whether blastp setting matches sequence formats.

• rje_blast_V2: Created.
→ Version 2.0: Initial Compilation from rje_blast_V1 V1.14.

• rje_db: Updated from Version 0.3.
→ Version 0.4: Improved use of AutoID and added Table.autoID() method.

• rje_ensembl: Updated from Version 2.8.
→ Version 2.9: Reduced DNA chromosome downloads. Updated some species data. Added "known_by_projection" handling.

• rje_genbank: Updated from Version 0.2.
→ Version 0.3: Added reloading of features.

• rje_hmm_V1: Created.
→ Version 0.0: Initial Working Compilation.
→ Version 1.0: Working version with multiple HMM capacity
→ Version 1.1: Added hmmpfam option
→ Version 1.2: Cleaned up and debugged for rje_ensembl.ensDat()

• rje_hmm_V2: Created.
→ Version 2.0: Initial HMMER3.0 version based on Version 1.2 and RJE_BLAST 2.0.

• rje_markov: Updated from Version 2.1.

• rje_mascot: Updated from Version 1.0.
→ Version 1.1: Fixed bugs for reading in data with unmatched peptides and iTRAQ data.
→ Version 1.2: Added

• rje_menu: Updated from Version 0.2.
→ Version 0.3: Modified to work with new object types.

• rje_ppi: Updated from Version 2.5.
→ Version 2.6: Added addPPI(hub,spoke,evidence) method. Added nodelist option.
→ Version 2.7: Added tabout=T/F Output PPI data as Node and Edge tables [False]

• rje_seqlist: Updated from Version 1.1.
→ Version 1.2: Added seqshuffle option for randomising sequences.

• rje_uniprot: Updated from Version 3.12.
→ Version 3.13: Minor bug fix for link table output.

• rje_xref: Created.
→ Version 0.0: Initial Compilation.

New SLiMSuite, SeqSuite and RJESuite downloads available

Just in time for Christmas, new releases of all the downloads are available at the Edwards Lab software page. Documentation is still lagging behind but will hopefully catch up (along with a bit of an overhaul of this blog). Questions welcome in the meantime.

In addition to QSLiMFinder 1.4, the biggest change this release is probably the upgrade of GOPHER. Version 3.x features improved organisation of output files for queries from different species in addition to a capacity to have several different multiple alignment programs run on the same orthologue sets. See the website for more info.

Updates since last release:

• gopher: Created.
→ Version 3.0: See archived GOPHER 1.9 and gopher_V2 2.9 for history and obselete options.
→ Version 3.0: Added organise=T/F and gopherdir=PATH for improved file organisation. Tightened savespace.
→ Version 3.0: Added compfilter=T/F for improved complexity filter and composition statistics control for *initial* BLAST.
→ Version 3.0: Changed default tree extension to *.nwk for compatibility with MEGA. Deleted _phosAlign() method.
→ Version 3.0: Added orthology ID option and alignment program to customise output further.
→ Version 3.1: Added full reciprocal best hit method. (fullrbh=T/F)

• gopher_V2: Updated from Version 2.8.
→ Version 2.9: Deleted oldStigg() method. Added simple Reciprocal Best Hit orthology prediction.

• qslimfinder: Updated from Version 1.2.
→ Version 1.3: Updated the output for Max/Min filtering and the pickup options.
→ Version 1.4: Added additional dictionary and list to store Query dimers and SLiMs for motif space calculations.
→ Version 1.4: Added qexact=T/F option for calculating Exact Query motif space (True) or estimating from dimers (False).

• slimfinder: Updated from Version 4.2.
→ Version 4.3: Updated the output for Max/Min filtering and the pickup options. Removed TempMaxSetting.
→ Version 4.4: Modified to work with GOPHER V3.0.

• rje: Updated from Version 4.3.
→ Version 4.4: Added lineFromIndex(target,file,re_index='^(\S+)\s',sortunique=False,xreplace=True).

• rje_seq: Updated from Version 3.13.
→ Version 3.14: Added CLUSTAL Omega alignment program ['clustalo']
→ Version 3.15: Added PAGAN alignment program ['pagan'] and (hopefully) fixed minor Windows fastacmd bug.

• rje_sequence: Updated from Version 2.1.
→ Version 2.2: Added more yeast species.

• rje_slimcalc: Updated from Version 0.4.
→ Version 0.5: Altered to use GOPHER V3 and handle nested alignment directories.

• rje_slimlist: Updated from Version 1.0.
→ Version 1.1: Modified to work with GOPHER V3.0 for alignments.

New SLiMSuite, SeqSuite and RJESuite releases are now available

New releases of SLiMSuite, SeqSuite and RJESuite are now available from the Edwards Lab software page.

Please note that the documentation (particularly the manuals) are still lagging a bit behind, so do report anything that does not make sense. The default settings also need to be verified as there is a chance that some of these may have inadvertently changed over the years. (The same core code is now used for the webservers, which often have different defaults.) Checking these along with updating and checking the servers themselves are ongoing priorities.

A full list of updated modules is given below. As well as SLiMMaker now handling end of sequence characters, the biggest changes this release are updates to CompariMotif to (3.7) output unmatched input motifs and (3.8) improve handling of partially overlapping ambiguous positions (e.g. [AGS] and [ST]). The motivation behind both these changes is the ongoing benchmarking (and preparation for publication) of QSLiMFinder and the creation of SLiMBench for benchmarking motif prediction methods. A QSLiMFinder section has been added to the SLiMFinder Manual (section 5.4). SLiMBench is still a work in progress and will be documented in a later release.

Updates since last release:

• comparimotif_V3: Updated from Version 3.6.
→ Version 3.7: Added coreIC and output of unmatched motifs.
→ Version 3.8: Added overlaps=T/F : Whether to include overlapping ambiguities (e.g. [KR] vs [HK]) as match [True]
→ Version 3.8: Changed scoring of overlapping ambiguities - uses IC of all possible ambiguities. Added "Ugly" match type.

• slimbench: Created.
→ Version 0.0: Initial Compilation.
→ Version 0.1: Functional version with benchmarking dataset generation.
→ Version 1.0: Consolidation of "working" version with additional basic benchmarking analysis.
→ Version 1.1: Added simulated dataset construction and benchmarking.
→ Version 1.2: Added MinIC filtering to benchmark assessment. Sorted beginning/end of line for reduced ELMs.
→ Version 1.3: Made SimCount a list rather than Integer. Sorted CompariMotif assessment issue.
→ Version 1.4: Added ICCut and SLiMLenCut as lists and output columns.
→ Version 1.5: Added Summary Results output table. Removed PropRes.

• slimmaker: Updated from Version 1.0.
→ Version 1.1: Modified to work with end of line characters.

• slimsearch: Updated from Version 1.5.
→ Version 1.6: Minor tweaks to Log output. Add option for UPC number in occ output.

• rje: Updated from Version 4.1.
→ Version 4.2: Modified INI reading across the board to look in ../settings/ and look for defaults.ini as well as rje.ini.
→ Version 4.2: Enabled handing on -ini FILE in addition to ini=FILE.
→ Version 4.3: Added ilist and nlist types to cmdRead for objects. (Lists of integers and floats). Add ratio() function.

• rje_blast: Updated from Version 1.13.
→ Version 1.14: Added blast.checkProg(qtype,stype) to check whether blastp setting matches sequence formats.

• rje_db: Created.
→ Version 0.0: Initial Compilation.
→ Version 0.1: Added merge tables option.
→ Version 0.2: Miscellaneous updates to various methods.
→ Version 0.3: Minor doc tweaks and added keepFields().

• rje_seq: Updated from Version 3.12.
→ Version 3.13: Updated sequence type checking for use with GABLAM 2.10.

• rje_seqlist: Created.
→ Version 0.0: Initial Compilation. Based on rje_seq 3.10.
→ Version 0.1: Added basic species filtering and sequence output.
→ Version 0.2: Added upper case filtering.
→ Version 0.3: Added accnum filtering and sequence renaming.
→ Version 0.4: Added sequence redundancy filtering.
→ Version 0.5: Added newgene=X for sequence renaming (newgene_spcode__newaccXXX). NewAcc no longer fixed Upper Case.
→ Version 1.0: Upgraded to "ready" Version 1.0. Added concatenate=T and split=X options for sequence concatenation.
→ Version 1.0: Added reading of sequence type from rje_seq.py and mixed=T/F.
→ Version 1.1: Added shortName() and modified SeqDict.

• rje_sequence: Updated from Version 2.0.
→ Version 2.1: Added re_unirefprot = re.compile('^([A-Za-z0-9\-]+)\s+([A-Za-z0-9]+)_([A-Za-z0-9]+)\s+')

• rje_slim: Updated from Version 1.5.
→ Version 1.6: Fixed splitting bug introduced by lower case motifs.

• rje_slimcore: Updated from Version 1.8.
→ Version 1.9: Minor modifications to Log output. Updated motifSeq() function to output unmasked sequences.

• rje_slimlist: Updated from Version 0.6.
→ Version 1.0: Functional module with lower case motif splitting fixed and ? -> .{0,1} replacement.

• rje_zen: Updated from Version 1.0.
→ Version 1.1: Added a few more words here and there.

Minor bug in CompariMotif 3.6 download

The recent fixing up of the server has highlighted a couple of bugs that have crept in during the upgrading of CompariMotif to handle complex motifs better. (More on this in a later post.) The problem is restricted to lower case motifs, so stick to upper case and all will be well. It will be fixed in the next release. A replacement rje_slimlist.py file is available on request if anyone needs to fix it urgently. (This also features replacement of "?" characters with "{0,1}" so these motifs are no longer rejected.)

New software downloads available

SLiMsuite and SeqSuite downloads have now been updated and are available from my homepage.

The only real change of note since the last release is that CompariMotif can now recognise ELM Class downloads and the motif splitting has been updated to cope with variable-length non-wildcard repeats (e.g. R{0,1}).

The CompariMotif webserver is undergoing some final checks and should be back up shortly, with SLiMFinder and SLiMSearch to follow.

New software downloads now available

Updated versions of all packages are now available for download. Unfortunately, due to limited time availability, the manuals are getting a little out of date with respect to all the available functions but the readme pages contain all the latest options and defaults. Please contact me if you find any bugs and/or want specific documentation improved. It is on the (long) list of things to try and get done over the summer!

Note that in a slight modification of previous releases, zip downloads now contain the creation date in the name (e.g. rjesuite.2012-07-26.tar.gz) and will be archived.

New software downloads now available

Updated software packages for SeqSuite, SLiMSuite and RJESuite are now available from the Edwards Lab software page. These downloads incorporate a variety of miscellaneous bug fixes and minor updates.

Websites back up

After some scheduled maintenance work that unfortunately took down the SeqSuite and SLiMSuite download and documentation pages, the website is now back up and running again.

Another new website is born!

Well, this is more of an update than a truly new site but the latest downloadable versions of the SeqSuite and SLiMSuite packages, along with improved (I hope!) websites and documentation are now available at: http://www.soton.ac.uk/~re1u06/software/packages/. This site is still a work in progress, so please send in comments and suggestions.