vsnp_get_snps: vsnp_build_tables.py annotate

annotate vsnp_build_tables.py @ 1:9ac0b1d5560d draft

"planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab"

author	iuc
date	Tue, 16 Nov 2021 20:11:30 +0000
parents	ec6e02f4eab7
children	6fdfdee72f8a

rev	line source
0 ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	1 #!/usr/bin/env python
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	2
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	3 import argparse
1 9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	4 import multiprocessing
0 ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	5 import os
1 9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	6 import queue
0 ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	7 import re
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	8
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	9 import pandas
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	10 import pandas.io.formats.excel
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	11 from Bio import SeqIO
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	12
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	13 # Maximum columns allowed in a LibreOffice
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	14 # spreadsheet is 1024. Excel allows for
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	15 # 16,384 columns, but we'll set the lower
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	16 # number as the maximum. Some browsers
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	17 # (e.g., Firefox on Linux) are configured
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	18 # to use LibreOffice for Excel spreadsheets.
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	19 MAXCOLS = 1024
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	20 OUTPUT_EXCEL_DIR = 'output_excel_dir'
1 9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	21 INPUT_JSON_AVG_MQ_DIR = 'input_json_avg_mq_dir'
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	22 INPUT_JSON_DIR = 'input_json_dir'
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	23 INPUT_NEWICK_DIR = 'input_newick_dir'
0 ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	24
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	25
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	26 def annotate_table(table_df, group, annotation_dict):
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	27 for gbk_chrome, pro in list(annotation_dict.items()):
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	28 ref_pos = list(table_df)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	29 ref_series = pandas.Series(ref_pos)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	30 ref_df = pandas.DataFrame(ref_series.str.split(':', expand=True).values, columns=['reference', 'position'])
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	31 all_ref = ref_df[ref_df['reference'] == gbk_chrome]
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	32 positions = all_ref.position.to_frame()
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	33 # Create an annotation file.
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	34 annotation_file = "%s_annotations.csv" % group
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	35 with open(annotation_file, "a") as fh:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	36 for _, row in positions.iterrows():
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	37 pos = row.position
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	38 try:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	39 aaa = pro.iloc[pro.index.get_loc(int(pos))][['chrom', 'locus', 'product', 'gene']]
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	40 try:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	41 chrom, name, locus, tag = aaa.values[0]
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	42 print("{}:{}\t{}, {}, {}".format(chrom, pos, locus, tag, name), file=fh)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	43 except ValueError:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	44 # If only one annotation for the entire
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	45 # chromosome (e.g., flu) then having [0] fails
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	46 chrom, name, locus, tag = aaa.values
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	47 print("{}:{}\t{}, {}, {}".format(chrom, pos, locus, tag, name), file=fh)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	48 except KeyError:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	49 print("{}:{}\tNo annotated product".format(gbk_chrome, pos), file=fh)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	50 # Read the annotation file into a data frame.
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	51 annotations_df = pandas.read_csv(annotation_file, sep='\t', header=None, names=['index', 'annotations'], index_col='index')
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	52 # Remove the annotation_file from disk since both
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	53 # cascade and sort tables are built using the file,
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	54 # and it is opened for writing in append mode.
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	55 os.remove(annotation_file)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	56 # Process the data.
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	57 table_df_transposed = table_df.T
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	58 table_df_transposed.index = table_df_transposed.index.rename('index')
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	59 table_df_transposed = table_df_transposed.merge(annotations_df, left_index=True, right_index=True)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	60 table_df = table_df_transposed.T
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	61 return table_df
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	62
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	63
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	64 def excel_formatter(json_file_name, excel_file_name, group, annotation_dict):
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	65 pandas.io.formats.excel.header_style = None
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	66 table_df = pandas.read_json(json_file_name, orient='split')
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	67 if annotation_dict is not None:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	68 table_df = annotate_table(table_df, group, annotation_dict)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	69 else:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	70 table_df = table_df.append(pandas.Series(name='no annotations'))
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	71 writer = pandas.ExcelWriter(excel_file_name, engine='xlsxwriter')
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	72 table_df.to_excel(writer, sheet_name='Sheet1')
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	73 writer_book = writer.book
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	74 ws = writer.sheets['Sheet1']
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	75 format_a = writer_book.add_format({'bg_color': '#58FA82'})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	76 format_g = writer_book.add_format({'bg_color': '#F7FE2E'})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	77 format_c = writer_book.add_format({'bg_color': '#0000FF'})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	78 format_t = writer_book.add_format({'bg_color': '#FF0000'})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	79 format_normal = writer_book.add_format({'bg_color': '#FDFEFE'})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	80 formatlowqual = writer_book.add_format({'font_color': '#C70039', 'bg_color': '#E2CFDD'})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	81 format_ambigous = writer_book.add_format({'font_color': '#C70039', 'bg_color': '#E2CFDD'})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	82 format_n = writer_book.add_format({'bg_color': '#E2CFDD'})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	83 rows, cols = table_df.shape
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	84 ws.set_column(0, 0, 30)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	85 ws.set_column(1, cols, 2.1)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	86 ws.freeze_panes(2, 1)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	87 format_annotation = writer_book.add_format({'font_color': '#0A028C', 'rotation': '-90', 'align': 'top'})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	88 # Set last row.
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	89 ws.set_row(rows + 1, cols + 1, format_annotation)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	90 # Make sure that row/column locations don't overlap.
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	91 ws.conditional_format(rows - 2, 1, rows - 1, cols, {'type': 'cell', 'criteria': '<', 'value': 55, 'format': formatlowqual})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	92 ws.conditional_format(2, 1, rows - 2, cols, {'type': 'cell', 'criteria': '==', 'value': 'B$2', 'format': format_normal})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	93 ws.conditional_format(2, 1, rows - 2, cols, {'type': 'text', 'criteria': 'containing', 'value': 'A', 'format': format_a})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	94 ws.conditional_format(2, 1, rows - 2, cols, {'type': 'text', 'criteria': 'containing', 'value': 'G', 'format': format_g})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	95 ws.conditional_format(2, 1, rows - 2, cols, {'type': 'text', 'criteria': 'containing', 'value': 'C', 'format': format_c})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	96 ws.conditional_format(2, 1, rows - 2, cols, {'type': 'text', 'criteria': 'containing', 'value': 'T', 'format': format_t})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	97 ws.conditional_format(2, 1, rows - 2, cols, {'type': 'text', 'criteria': 'containing', 'value': 'S', 'format': format_ambigous})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	98 ws.conditional_format(2, 1, rows - 2, cols, {'type': 'text', 'criteria': 'containing', 'value': 'Y', 'format': format_ambigous})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	99 ws.conditional_format(2, 1, rows - 2, cols, {'type': 'text', 'criteria': 'containing', 'value': 'R', 'format': format_ambigous})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	100 ws.conditional_format(2, 1, rows - 2, cols, {'type': 'text', 'criteria': 'containing', 'value': 'W', 'format': format_ambigous})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	101 ws.conditional_format(2, 1, rows - 2, cols, {'type': 'text', 'criteria': 'containing', 'value': 'K', 'format': format_ambigous})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	102 ws.conditional_format(2, 1, rows - 2, cols, {'type': 'text', 'criteria': 'containing', 'value': 'M', 'format': format_ambigous})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	103 ws.conditional_format(2, 1, rows - 2, cols, {'type': 'text', 'criteria': 'containing', 'value': 'N', 'format': format_n})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	104 ws.conditional_format(2, 1, rows - 2, cols, {'type': 'text', 'criteria': 'containing', 'value': '-', 'format': format_n})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	105 format_rotation = writer_book.add_format({})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	106 format_rotation.set_rotation(90)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	107 for column_num, column_name in enumerate(list(table_df.columns)):
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	108 ws.write(0, column_num + 1, column_name, format_rotation)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	109 format_annotation = writer_book.add_format({'font_color': '#0A028C', 'rotation': '-90', 'align': 'top'})
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	110 # Set last row.
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	111 ws.set_row(rows, 400, format_annotation)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	112 writer.save()
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	113
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	114
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	115 def get_annotation_dict(gbk_file):
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	116 gbk_dict = SeqIO.to_dict(SeqIO.parse(gbk_file, "genbank"))
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	117 annotation_dict = {}
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	118 tmp_file = "features.csv"
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	119 # Create a file of chromosomes and features.
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	120 for chromosome in list(gbk_dict.keys()):
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	121 with open(tmp_file, 'w+') as fh:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	122 for feature in gbk_dict[chromosome].features:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	123 if "CDS" in feature.type or "rRNA" in feature.type:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	124 try:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	125 product = feature.qualifiers['product'][0]
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	126 except KeyError:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	127 product = None
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	128 try:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	129 locus = feature.qualifiers['locus_tag'][0]
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	130 except KeyError:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	131 locus = None
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	132 try:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	133 gene = feature.qualifiers['gene'][0]
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	134 except KeyError:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	135 gene = None
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	136 fh.write("%s\t%d\t%d\t%s\t%s\t%s\n" % (chromosome, int(feature.location.start), int(feature.location.end), locus, product, gene))
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	137 # Read the chromosomes and features file into a data frame.
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	138 df = pandas.read_csv(tmp_file, sep='\t', names=["chrom", "start", "stop", "locus", "product", "gene"])
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	139 # Process the data.
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	140 df = df.sort_values(['start', 'gene'], ascending=[True, False])
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	141 df = df.drop_duplicates('start')
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	142 pro = df.reset_index(drop=True)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	143 pro.index = pandas.IntervalIndex.from_arrays(pro['start'], pro['stop'], closed='both')
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	144 annotation_dict[chromosome] = pro
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	145 return annotation_dict
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	146
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	147
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	148 def get_sample_name(file_path):
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	149 base_file_name = os.path.basename(file_path)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	150 if base_file_name.find(".") > 0:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	151 # Eliminate the extension.
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	152 return os.path.splitext(base_file_name)[0]
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	153 return base_file_name
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	154
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	155
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	156 def output_cascade_table(cascade_order, mqdf, group, annotation_dict):
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	157 cascade_order_mq = pandas.concat([cascade_order, mqdf], join='inner')
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	158 output_table(cascade_order_mq, "cascade", group, annotation_dict)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	159
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	160
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	161 def output_excel(df, type_str, group, annotation_dict, count=None):
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	162 # Output the temporary json file that
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	163 # is used by the excel_formatter.
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	164 if count is None:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	165 if group is None:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	166 json_file_name = os.path.join(OUTPUT_EXCEL_DIR, "%s_order_mq.json" % type_str)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	167 excel_file_name = os.path.join(OUTPUT_EXCEL_DIR, "%s_table.xlsx" % type_str)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	168 else:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	169 json_file_name = os.path.join(OUTPUT_EXCEL_DIR, "%s_%s_order_mq.json" % (group, type_str))
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	170 excel_file_name = os.path.join(OUTPUT_EXCEL_DIR, "%s_%s_table.xlsx" % (group, type_str))
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	171 else:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	172 # The table has more columns than is allowed by the
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	173 # MAXCOLS setting, so multiple files will be produced
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	174 # as an output collection.
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	175 if group is None:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	176 json_file_name = os.path.join(OUTPUT_EXCEL_DIR, "%s_order_mq_%d.json" % (type_str, count))
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	177 excel_file_name = os.path.join(OUTPUT_EXCEL_DIR, "%s_table_%d.xlsx" % (type_str, count))
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	178 else:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	179 json_file_name = os.path.join(OUTPUT_EXCEL_DIR, "%s_%s_order_mq_%d.json" % (group, type_str, count))
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	180 excel_file_name = os.path.join(OUTPUT_EXCEL_DIR, "%s_%s_table_%d.xlsx" % (group, type_str, count))
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	181 df.to_json(json_file_name, orient='split')
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	182 # Output the Excel file.
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	183 excel_formatter(json_file_name, excel_file_name, group, annotation_dict)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	184
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	185
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	186 def output_sort_table(cascade_order, mqdf, group, annotation_dict):
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	187 sort_df = cascade_order.T
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	188 sort_df['abs_value'] = sort_df.index
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	189 sort_df[['chrom', 'pos']] = sort_df['abs_value'].str.split(':', expand=True)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	190 sort_df = sort_df.drop(['abs_value', 'chrom'], axis=1)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	191 sort_df.pos = sort_df.pos.astype(int)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	192 sort_df = sort_df.sort_values(by=['pos'])
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	193 sort_df = sort_df.drop(['pos'], axis=1)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	194 sort_df = sort_df.T
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	195 sort_order_mq = pandas.concat([sort_df, mqdf], join='inner')
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	196 output_table(sort_order_mq, "sort", group, annotation_dict)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	197
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	198
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	199 def output_table(df, type_str, group, annotation_dict):
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	200 if isinstance(group, str) and group.startswith("dataset"):
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	201 # Inputs are single files, not collections,
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	202 # so input file names are not useful for naming
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	203 # output files.
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	204 group_str = None
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	205 else:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	206 group_str = group
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	207 count = 0
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	208 chunk_start = 0
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	209 chunk_end = 0
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	210 column_count = df.shape[1]
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	211 if column_count >= MAXCOLS:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	212 # Here the number of columns is greater than
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	213 # the maximum allowed by Excel, so multiple
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	214 # outputs will be produced.
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	215 while column_count >= MAXCOLS:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	216 count += 1
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	217 chunk_end += MAXCOLS
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	218 df_of_type = df.iloc[:, chunk_start:chunk_end]
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	219 output_excel(df_of_type, type_str, group_str, annotation_dict, count=count)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	220 chunk_start += MAXCOLS
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	221 column_count -= MAXCOLS
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	222 count += 1
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	223 df_of_type = df.iloc[:, chunk_start:]
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	224 output_excel(df_of_type, type_str, group_str, annotation_dict, count=count)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	225 else:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	226 output_excel(df, type_str, group_str, annotation_dict)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	227
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	228
1 9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	229 def preprocess_tables(task_queue, annotation_dict, timeout):
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	230 while True:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	231 try:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	232 tup = task_queue.get(block=True, timeout=timeout)
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	233 except queue.Empty:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	234 break
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	235 newick_file, json_file, json_avg_mq_file = tup
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	236 avg_mq_series = pandas.read_json(json_avg_mq_file, typ='series', orient='split')
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	237 # Map quality to dataframe.
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	238 mqdf = avg_mq_series.to_frame(name='MQ')
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	239 mqdf = mqdf.T
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	240 # Get the group.
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	241 group = get_sample_name(newick_file)
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	242 snps_df = pandas.read_json(json_file, orient='split')
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	243 with open(newick_file, 'r') as fh:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	244 for line in fh:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	245 line = re.sub('[:,]', '\n', line)
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	246 line = re.sub('[)(]', '', line)
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	247 line = re.sub(r'[0-9].\.[0-9].\n', '', line)
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	248 line = re.sub('root\n', '', line)
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	249 sample_order = line.split('\n')
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	250 sample_order = list([_f for _f in sample_order if _f])
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	251 sample_order.insert(0, 'root')
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	252 tree_order = snps_df.loc[sample_order]
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	253 # Count number of SNPs in each column.
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	254 snp_per_column = []
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	255 for column_header in tree_order:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	256 count = 0
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	257 column = tree_order[column_header]
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	258 for element in column:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	259 if element != column[0]:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	260 count = count + 1
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	261 snp_per_column.append(count)
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	262 row1 = pandas.Series(snp_per_column, tree_order.columns, name="snp_per_column")
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	263 # Count number of SNPS from the
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	264 # top of each column in the table.
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	265 snp_from_top = []
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	266 for column_header in tree_order:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	267 count = 0
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	268 column = tree_order[column_header]
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	269 # for each element in the column
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	270 # skip the first element
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	271 for element in column[1:]:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	272 if element == column[0]:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	273 count = count + 1
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	274 else:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	275 break
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	276 snp_from_top.append(count)
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	277 row2 = pandas.Series(snp_from_top, tree_order.columns, name="snp_from_top")
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	278 tree_order = tree_order.append([row1])
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	279 tree_order = tree_order.append([row2])
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	280 # In pandas=0.18.1 even this does not work:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	281 # abc = row1.to_frame()
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	282 # abc = abc.T --> tree_order.shape (5, 18), abc.shape (1, 18)
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	283 # tree_order.append(abc)
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	284 # Continue to get error: "*** ValueError: all the input arrays must have same number of dimensions"
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	285 tree_order = tree_order.T
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	286 tree_order = tree_order.sort_values(['snp_from_top', 'snp_per_column'], ascending=[True, False])
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	287 tree_order = tree_order.T
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	288 # Remove snp_per_column and snp_from_top rows.
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	289 cascade_order = tree_order[:-2]
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	290 # Output the cascade table.
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	291 output_cascade_table(cascade_order, mqdf, group, annotation_dict)
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	292 # Output the sorted table.
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	293 output_sort_table(cascade_order, mqdf, group, annotation_dict)
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	294 task_queue.task_done()
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	295
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	296
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	297 def set_num_cpus(num_files, processes):
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	298 num_cpus = int(multiprocessing.cpu_count())
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	299 if num_files < num_cpus and num_files < processes:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	300 return num_files
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	301 if num_cpus < processes:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	302 half_cpus = int(num_cpus / 2)
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	303 if num_files < half_cpus:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	304 return num_files
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	305 return half_cpus
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	306 return processes
0 ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	307
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	308
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	309 if __name__ == '__main__':
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	310 parser = argparse.ArgumentParser()
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	311
1 9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	312 parser.add_argument('--input_avg_mq_json', action='store', dest='input_avg_mq_json', required=False, default=None, help='Average MQ json file')
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	313 parser.add_argument('--input_newick', action='store', dest='input_newick', required=False, default=None, help='Newick file')
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	314 parser.add_argument('--input_snps_json', action='store', dest='input_snps_json', required=False, default=None, help='SNPs json file')
0 ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	315 parser.add_argument('--gbk_file', action='store', dest='gbk_file', required=False, default=None, help='Optional gbk file'),
1 9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	316 parser.add_argument('--processes', action='store', dest='processes', type=int, help='User-selected number of processes to use for job splitting')
0 ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	317
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	318 args = parser.parse_args()
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	319
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	320 if args.gbk_file is not None:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	321 # Create the annotation_dict for annotating
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	322 # the Excel tables.
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	323 annotation_dict = get_annotation_dict(args.gbk_file)
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	324 else:
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	325 annotation_dict = None
ec6e02f4eab7 "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 95b221f68d19702681babd765c67caeeb24e7f1d" iuc parents: diff changeset	326
1 9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	327 # The assumption here is that the list of files
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	328 # in both INPUT_NEWICK_DIR and INPUT_JSON_DIR are
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	329 # named such that they are properly matched if
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	330 # the directories contain more than 1 file (i.e.,
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	331 # hopefully the newick file names and json file names
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	332 # will be something like Mbovis-01D6_* so they can be
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	333 # sorted and properly associated with each other).
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	334 if args.input_newick is not None:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	335 newick_files = [args.input_newick]
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	336 else:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	337 newick_files = []
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	338 for file_name in sorted(os.listdir(INPUT_NEWICK_DIR)):
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	339 file_path = os.path.abspath(os.path.join(INPUT_NEWICK_DIR, file_name))
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	340 newick_files.append(file_path)
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	341 if args.input_snps_json is not None:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	342 json_files = [args.input_snps_json]
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	343 else:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	344 json_files = []
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	345 for file_name in sorted(os.listdir(INPUT_JSON_DIR)):
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	346 file_path = os.path.abspath(os.path.join(INPUT_JSON_DIR, file_name))
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	347 json_files.append(file_path)
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	348 if args.input_avg_mq_json is not None:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	349 json_avg_mq_files = [args.input_avg_mq_json]
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	350 else:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	351 json_avg_mq_files = []
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	352 for file_name in sorted(os.listdir(INPUT_JSON_AVG_MQ_DIR)):
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	353 file_path = os.path.abspath(os.path.join(INPUT_JSON_AVG_MQ_DIR, file_name))
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	354 json_avg_mq_files.append(file_path)
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	355
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	356 multiprocessing.set_start_method('spawn')
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	357 queue1 = multiprocessing.JoinableQueue()
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	358 queue2 = multiprocessing.JoinableQueue()
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	359 num_files = len(newick_files)
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	360 cpus = set_num_cpus(num_files, args.processes)
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	361 # Set a timeout for get()s in the queue.
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	362 timeout = 0.05
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	363
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	364 for i, newick_file in enumerate(newick_files):
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	365 json_file = json_files[i]
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	366 json_avg_mq_file = json_avg_mq_files[i]
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	367 queue1.put((newick_file, json_file, json_avg_mq_file))
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	368
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	369 # Complete the preprocess_tables task.
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	370 processes = [multiprocessing.Process(target=preprocess_tables, args=(queue1, annotation_dict, timeout, )) for _ in range(cpus)]
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	371 for p in processes:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	372 p.start()
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	373 for p in processes:
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	374 p.join()
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	375 queue1.join()
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	376
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	377 if queue1.empty():
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	378 queue1.close()
9ac0b1d5560d "planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/tools/vsnp commit 2a94c64d6c7236550bf483d2ffc4e86248c63aab" iuc parents: 0 diff changeset	379 queue1.join_thread()

Mercurial > repos > iuc > vsnp_get_snps

annotate vsnp_build_tables.py @ 1:9ac0b1d5560d draft