Mercurial > repos > iuc > data_manager_bakta

annotate data_manager/bakta_build_database.py @ 4:d74850cf4e42 draft

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
author iuc
date Fri, 25 Aug 2023 23:34:04 +0000
parents 3e73c97f025d
children baceff842902
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
1 import argparse
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
2 import hashlib
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
3 import json
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
4 import os
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
5 import re
4
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
6 import shutil
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
7 import sys
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
8 import tarfile
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
9 from datetime import datetime
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
10 from pathlib import Path
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
11
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
12
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
13 import requests
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
14
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
15
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
16 class GetBaktaDatabaseInfo:
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
17 """
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
18 Extract bakta database information to make a json file for data_manager
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
19 """
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
20
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
21 def __init__(
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
22 self,
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
23 data_table_name="bakta_database",
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
24 db_version="latest",
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
25 tarball_name="db.tar.gz",
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
26 test_mode=False,
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
27 ):
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
28 self.bakta_table_list = None
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
29 self.db_url = None
4
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
30 self.db_name = "bakta-db"
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
31 self.db_type = ""
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
32 self.data_table_entry = None
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
33 self.data_table_name = data_table_name
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
34 self.tar_name = tarball_name
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
35 self.db_version = db_version
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
36 self.DB_VERSIONS_URL = "https://raw.githubusercontent.com/oschwengers/bakta/master/db-versions.json"
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
37 self.DB_TEST_URL = "https://zenodo.org/record/8021032/files/db-versions.json"
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
38 self.test_mode = test_mode
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
39
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
40 def get_database_type(self):
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
41 self.light_db = bool(re.search(pattern="light", string=self.db_version))
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
42 self.db_version = self.db_version.split(sep="_")[0]
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
43 if self.light_db:
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
44 self.db_type = "light"
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
45 self.tar_name = "db-light.tar.gz"
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
46 self.md5 = self.fetch_db_versions()["md5-light"]
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
47 else:
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
48 self.md5 = self.fetch_db_versions()["md5"]
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
49
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
50 def get_data_table_format(self):
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
51 """
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
52 Skeleton of a data_table format
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
53 return: a data table formated for json output
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
54 """
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
55 self.data_table_entry = {"data_tables": {self.data_table_name: {}}}
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
56 return self.data_table_entry
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
57
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
58 def fetch_db_versions(self):
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
59 """
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
60 List bakta database info related to the db_version selected
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
61 """
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
62
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
63 if self.test_mode:
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
64 self.DB_VERSIONS_URL = self.DB_TEST_URL
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
65 try:
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
66 with requests.get(self.DB_VERSIONS_URL) as resp:
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
67 versions = json.loads(resp.content)
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
68 except IOError as e:
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
69 print(e, file=sys.stderr)
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
70 raise e
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
71
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
72 if self.db_version == "latest":
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
73 db_date_list = []
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
74 for db_dic in versions:
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
75 db_date_list.append(
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
76 datetime.strptime(db_dic["date"], "%Y-%m-%d").date()
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
77 )
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
78 filtered_version = max(versions, key=lambda x: x["date"])
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
79 else:
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
80 filtered_version = None
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
81 for item in versions:
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
82 if "{0}.{1}".format(item["major"], item["minor"]) == self.db_version:
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
83 filtered_version = item
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
84 break
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
85 if filtered_version is None:
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
86 print("No matching version detected in the list")
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
87 else:
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
88 self.db_url = f"https://zenodo.org/record/{filtered_version['record']}/files/{self.tar_name}"
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
89 return filtered_version
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
90
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
91 def get_data_manager(self, bakta_database_info):
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
92 self.bakta_table_list = self.get_data_table_format()
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
93 bakta_name = (
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
94 f"V{bakta_database_info['major']}."
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
95 f"{bakta_database_info['minor']}{self.db_type}_"
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
96 f"{bakta_database_info['date']}"
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
97 )
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
98 tool_version = str(
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
99 f"{bakta_database_info['software-min']['major']}."
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
100 f"{bakta_database_info['software-min']['minor']}"
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
101 )
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
102 data_info = dict(
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
103 value=bakta_name,
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
104 dbkey=bakta_database_info["record"],
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
105 bakta_version=tool_version,
4
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
106 path=self.db_name,
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
107 )
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
108 self.bakta_table_list["data_tables"][self.data_table_name] = [data_info]
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
109 return self.bakta_table_list
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
110
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
111
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
112 class InstallBaktaDatabase(GetBaktaDatabaseInfo):
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
113 """
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
114 Download the bakta database,
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
115 check md5 sum,
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
116 untar the download db and update for the amrfinderplus database
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
117 """
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
118
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
119 def __init__(
4
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
120 self,
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
121 db_dir=Path.cwd(),
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
122 db_name="bakta-db",
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
123 db_version="latest",
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
124 test_mode=False
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
125 ):
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
126 super().__init__()
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
127 self.md5 = None
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
128 self.db_version = db_version
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
129 self.db_dir = db_dir
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
130 self.db_name = db_name
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
131 self.tarball_path = ""
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
132 self.test_mode = test_mode
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
133 self.get_database_type()
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
134
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
135 def download(self):
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
136 bakta_path = Path(self.db_dir).joinpath(self.tar_name)
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
137 try:
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
138 with bakta_path.open("wb") as fh_out, requests.get(
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
139 self.db_url, stream=True) as resp:
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
140 total_length = resp.headers.get("content-length")
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
141 if total_length is None: # no content length header
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
142 for data in resp.iter_content(chunk_size=1024 * 1024):
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
143 fh_out.write(data)
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
144 else:
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
145 for data in resp.iter_content(chunk_size=1024 * 1024):
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
146 fh_out.write(data)
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
147 print(f"Download bakta database {self.db_version}")
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
148 self.tarball_path = bakta_path
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
149 except IOError:
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
150 print(
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
151 f"ERROR: Could not download file from Zenodo!"
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
152 f" url={self.db_url}, to={self.tarball_path}"
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
153 )
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
154
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
155 def untar(self):
4
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
156 db_path = Path(self.db_dir).joinpath(self.db_name)
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
157 try:
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
158 with self.tarball_path.open("rb") as fh_in, tarfile.open(
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
159 fileobj=fh_in, mode="r:gz"
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
160 ) as tar_file:
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
161 tar_file.extractall(path=db_path)
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
162 print(f"Untar the database in {db_path}")
4
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
163
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
164 if not self.test_mode:
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
165 self.moove_files(db_path=db_path)
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
166
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
167 except OSError:
4
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
168 sys.exit(f"ERROR: Could not extract {self.tar_name} " f"to {db_path}")
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
169
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
170 def moove_files(self, db_path):
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
171 if os.path.isdir(db_path.joinpath("db-light")):
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
172 input_dir = db_path.joinpath("db-light")
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
173 elif os.path.isdir(db_path.joinpath("db")):
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
174 input_dir = db_path.joinpath("db")
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
175 file_list = os.listdir(input_dir)
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
176 output_dir = db_path
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
177 for file in file_list:
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
178 input = input_dir.joinpath(file)
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
179 output = output_dir.joinpath(file)
d74850cf4e42 planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 404dc5eae30884c5814e463921eb3678bbd6878e
iuc
parents: 3
diff changeset
180 shutil.move(input, output)
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
181
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
182 def calc_md5_sum(self, buffer_size=1048576):
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
183 tarball_path = Path(self.db_dir).joinpath(self.tar_name)
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
184 md5 = hashlib.md5()
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
185 with tarball_path.open("rb") as fh:
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
186 data = fh.read(buffer_size)
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
187 while data:
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
188 md5.update(data)
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
189 data = fh.read(buffer_size)
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
190 if md5.hexdigest() == self.md5:
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
191 print("\t...md5 control database OK")
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
192 else:
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
193 print(
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
194 f"Error: corrupt database file! "
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
195 f"calculated md5 = {md5.hexdigest()}"
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
196 f" different from {self.md5} "
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
197 )
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
198
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
199
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
200 def parse_arguments():
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
201 # parse options and arguments
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
202 arg_parser = argparse.ArgumentParser()
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
203 arg_parser.add_argument("data_manager_json")
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
204 arg_parser.add_argument(
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
205 "-d",
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
206 "--database_version",
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
207 help="Select the database version "
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
208 "(major and minor eg. 4.0),"
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
209 "default is the latest version",
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
210 default="latest",
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
211 required=True,
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
212 )
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
213 arg_parser.add_argument(
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
214 "-t",
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
215 "--test",
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
216 action="store_true",
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
217 help="option to test the script with an empty database",
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
218 )
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
219 return arg_parser.parse_args()
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
220
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
221
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
222 def main():
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
223 all_args = parse_arguments()
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
224 with open(all_args.data_manager_json) as fh:
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
225 params = json.load(fh)
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
226 target_dir = params["output_data"][0]["extra_files_path"]
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
227 os.makedirs(target_dir)
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
228 # init the class to download bakta db
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
229 bakta_upload = InstallBaktaDatabase(
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
230 test_mode=all_args.test, db_version=all_args.database_version
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
231 )
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
232 bakta_db = bakta_upload.fetch_db_versions()
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
233 # update the path for galaxy
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
234 bakta_upload.db_dir = target_dir
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
235 # download the database
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
236 bakta_upload.download()
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
237 # check md5 sum
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
238 bakta_upload.calc_md5_sum()
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
239 # untar db
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
240 bakta_upload.untar()
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
241 # make the data_manager metadata
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
242 bakta_data_manager = bakta_upload.get_data_manager(bakta_database_info=bakta_db)
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
243 with open(all_args.data_manager_json, "w") as fh:
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
244 json.dump(bakta_data_manager, fh, sort_keys=True)
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
245
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
246
3
3e73c97f025d planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_build_bakta_database commit 487cb35fe55883ac6eeb8dda58b56c9ca2ec0a85
iuc
parents: 1
diff changeset
247 if __name__ == "__main__":
0
a19189a128cb planemo upload for repository https://github.com/galaxyproject/tools-iuc/tree/master/data_managers/data_manager_bakta commit fba6deae1d3707e0c14202433d0495e157745afd
iuc
parents:
diff changeset
248 main()