Mercurial > repos > tduigou > seq_to_db

annotate save_to_db.py @ 2:dad2c4c3450a draft default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
author tduigou
date Wed, 23 Jul 2025 09:44:33 +0000
parents 3daf04425ea1
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
1 import subprocess
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
2 import argparse
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
3 import time
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
4 import os
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
5 import socket
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
6 import re
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
7 import json
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
8 from sqlalchemy import create_engine, inspect
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
9 from sqlalchemy.engine.url import make_url
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
10 from sqlalchemy.sql import text
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
11 from sqlalchemy.exc import OperationalError
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
12
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
13
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
14 def fix_db_uri(uri):
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
15 """Replace __at__ with @ in the URI if needed."""
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
16 return uri.replace("__at__", "@")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
17
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
18
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
19 def is_port_in_use(uri):
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
20 """Check if a TCP port is already in use on host."""
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
21 url = make_url(uri)
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
22 host = url.host
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
23 port = url.port
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
24 with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
25 s.settimeout(2)
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
26 return s.connect_ex((host, port)) == 0
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
27
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
28 def extract_db_name(uri):
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
29 """Extract the database name from the SQLAlchemy URI."""
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
30 url = make_url(uri)
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
31 return url.database
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
32
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
33
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
34 # this fuction is to activate the Docker id the DB is in container. BUT IT IS NOT USED IN MAIN()
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
35 def start_postgres_container(db_name):
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
36 """Start a PostgreSQL container with the given database name as the container name."""
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
37 container_name = db_name
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
38
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
39 # Check if container is already running
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
40 container_running = subprocess.run(
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
41 f"docker ps -q -f name={container_name}", shell=True, capture_output=True, text=True
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
42 )
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
43
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
44 if container_running.stdout.strip():
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
45 print(f"Container '{container_name}' is already running.")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
46 return
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
47
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
48 # Check if container exists (stopped)
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
49 container_exists = subprocess.run(
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
50 f"docker ps -a -q -f name={container_name}", shell=True, capture_output=True, text=True
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
51 )
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
52
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
53 if container_exists.stdout.strip():
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
54 print(f"Starting existing container '{container_name}'...")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
55 subprocess.run(f"docker start {container_name}", shell=True)
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
56 print(f"PostgreSQL Docker container '{container_name}' activated.")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
57 return
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
58
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
59 # If container does not exist, create and start a new one
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
60 port = 5432 if not is_port_in_use(5432) else 5433
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
61 postgres_password = os.getenv("POSTGRES_PASSWORD", "RK17")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
62
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
63 start_command = [
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
64 "docker", "run", "--name", container_name,
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
65 "-e", f"POSTGRES_PASSWORD={postgres_password}",
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
66 "-p", f"{port}:5432",
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
67 "-d", "postgres"
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
68 ]
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
69
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
70 try:
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
71 subprocess.run(start_command, check=True)
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
72 print(f"PostgreSQL Docker container '{container_name}' started on port {port}.")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
73 except subprocess.CalledProcessError as e:
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
74 print(f"Failed to start Docker container: {e}")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
75
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
76
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
77 def wait_for_db(uri, timeout=60):
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
78 """Try connecting to the DB until it works or timeout."""
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
79 engine = create_engine(uri)
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
80 start_time = time.time()
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
81 while time.time() - start_time < timeout:
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
82 try:
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
83 with engine.connect():
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
84 print("Connected to database.")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
85 return
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
86 except OperationalError:
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
87 print("Database not ready, retrying...")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
88 time.sleep(2)
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
89 raise Exception("Database connection failed after timeout.")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
90
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
91
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
92 def push_gb_annotations(gb_files, sequence_column, annotation_column, db_uri, table_name, fragment_column_name, output, file_name_mapping):
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
93 """Push GenBank file content into the database if the fragment is not already present."""
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
94 db_uri = fix_db_uri(db_uri)
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
95 engine = create_engine(db_uri)
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
96 inserted_fragments = []
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
97
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
98 try:
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
99 # Parse the file_name_mapping string into a dictionary {base_file_name: fragment_name}
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
100 file_name_mapping_dict = {
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
101 os.path.basename(path): os.path.splitext(fragment_name)[0]
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
102 for mapping in file_name_mapping.split(",")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
103 for path, fragment_name in [mapping.split(":")]
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
104 }
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
105
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
106 #print("File name mapping dictionary:")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
107 #print(file_name_mapping_dict) # Debugging: Print the mapping dictionary
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
108
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
109 with engine.begin() as connection:
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
110 inspector = inspect(engine)
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
111 columns = [col['name'] for col in inspector.get_columns(table_name)]
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
112
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
113 if fragment_column_name not in columns:
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
114 raise ValueError(f"Fragment column '{fragment_column_name}' not found in table '{table_name}'.")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
115
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
116 # Get existing fragments
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
117 all_rows = connection.execute(text(f"SELECT {fragment_column_name} FROM {table_name}")).fetchall()
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
118 existing_fragments = {row[0] for row in all_rows}
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
119
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
120 insert_rows = []
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
121
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
122 for gb_file in gb_files:
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
123 # Extract base file name (just the file name, not the full path)
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
124 real_file_name = os.path.basename(gb_file)
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
125 fragment_name = file_name_mapping_dict.get(real_file_name)
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
126
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
127 print(f"Processing file: {real_file_name}({fragment_name})") # Debugging: Log the current file
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
128
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
129 # Get the corresponding fragment name from the mapping
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
130 fragment_name = file_name_mapping_dict.get(real_file_name)
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
131
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
132 if not fragment_name:
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
133 raise ValueError(f"Fragment name not found for file '{real_file_name}' in file_name_mapping.")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
134
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
135 # If the fragment is already in the DB, raise an error and stop the process
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
136 if fragment_name in existing_fragments:
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
137 raise RuntimeError(f"Fatal Error: Fragment '{fragment_name}' already exists in DB. Stopping the process.")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
138
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
139 with open(gb_file, "r") as f:
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
140 content = f.read()
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
141
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
142 origin_match = re.search(r"^ORIGIN.*$", content, flags=re.MULTILINE)
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
143 if not origin_match:
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
144 raise ValueError(f"ORIGIN section not found in file: {gb_file}")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
145
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
146 origin_start = origin_match.start()
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
147 annotation_text = content[:origin_start].strip()
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
148 sequence_text = content[origin_start:].strip()
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
149
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
150 values = {}
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
151 values[fragment_column_name] = fragment_name
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
152 values[annotation_column] = annotation_text
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
153 values[sequence_column] = sequence_text
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
154
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
155 insert_rows.append(values)
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
156 inserted_fragments.append(fragment_name)
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
157
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
158 # Insert the rows into the database
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
159 for values in insert_rows:
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
160 col_names = ", ".join(values.keys())
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
161 placeholders = ", ".join([f":{key}" for key in values.keys()])
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
162 insert_stmt = text(f"INSERT INTO {table_name} ({col_names}) VALUES ({placeholders})")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
163
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
164 # print(f"Inserting into DB: {values}") # Debugging print statement
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
165 connection.execute(insert_stmt, values)
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
166
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
167 # print(f"Insert result: {result.rowcount if hasattr(result, 'rowcount') else 'N/A'}") # Debugging the row count
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
168
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
169 print(f"Inserted {len(insert_rows)} fragments.")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
170
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
171 # Write inserted fragment names to a text file
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
172 with open(output, "w") as log_file:
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
173 for frag in inserted_fragments:
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
174 log_file.write(f"{frag}\n")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
175 print(f"Fragment names written to '{output}'.")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
176
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
177 except Exception as e:
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
178 print(f"Error during GB file insertion: {e}")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
179 raise
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
180
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
181
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
182 def main():
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
183 parser = argparse.ArgumentParser(description="Fetch annotations from PostgreSQL database and save as JSON.")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
184 parser.add_argument("--input", required=True, help="Input gb files")
2
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
185 parser.add_argument("--use_json_paramers", required=False, help="Use parameters from JSON: true/false")
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
186 parser.add_argument("--sequence_column", required=False, help="DB column contains sequence for GenBank file")
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
187 parser.add_argument("--annotation_column", required=False, help="DB column contains head for GenBank file")
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
188 parser.add_argument("--db_uri", required=False, help="Database URI connection string")
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
189 parser.add_argument("--table", required=False, help="Table name in the database")
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
190 parser.add_argument("--fragment_column", required=False, help="Fragment column name in the database")
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
191 parser.add_argument("--output", required=False, help="Text report")
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
192 parser.add_argument("--file_name_mapping", required=False, help="Real fragment names")
0
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
193 parser.add_argument("--json_conf", required=False, help="JSON config file with DB parameters")
2
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
194 parser.add_argument("--execution_enable", required=False, help="Enable or disable execution")
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
195 parser.add_argument("--json_generating", required=False, help="Generate JSON: true/false")
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
196 parser.add_argument("--json_output", required=False, help="Output path for generated JSON")
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
197
0
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
198 args = parser.parse_args()
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
199
1
3daf04425ea1 planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents: 0
diff changeset
200 if args.execution_enable == 'false':
2
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
201 print("Execution disabled. 'Send Request to DB' is set to 'false'")
1
3daf04425ea1 planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents: 0
diff changeset
202 return
3daf04425ea1 planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents: 0
diff changeset
203
2
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
204 config_params = {}
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
205 use_json = args.use_json_paramers == 'true'
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
206 generate_json = args.json_generating == 'true'
0
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
207
2
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
208 if use_json:
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
209 if not args.json_conf:
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
210 raise ValueError("You must provide --json_conf when --use_json_paramers is 'true'")
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
211 with open(args.json_conf, "r") as f:
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
212 config_params = json.load(f)
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
213 if config_params.get("execution", "") == "false":
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
214 print("Execution blocked by config (execution = false)")
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
215 return
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
216 else:
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
217 config_params = {
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
218 "table": args.table,
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
219 "sequence_column": args.sequence_column,
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
220 "annotation_column": args.annotation_column,
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
221 "fragment_column": args.fragment_column,
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
222 "db_uri": args.db_uri,
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
223 "execution": args.execution_enable
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
224 }
0
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
225
2
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
226 if generate_json:
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
227 if not args.json_output:
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
228 raise ValueError("You must provide --json_output when --json_generating is 'true'")
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
229 with open(args.json_output, "w") as f:
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
230 json.dump(config_params, f, indent=2)
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
231 print(f"JSON configuration written to: {args.json_output}")
0
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
232
2
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
233 # Extract final resolved parameters
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
234 table = config_params["table"]
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
235 sequence_column = config_params["sequence_column"]
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
236 annotation_column = config_params["annotation_column"]
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
237 fragment_column = config_params["fragment_column"]
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
238 db_uri = fix_db_uri(config_params["db_uri"])
0
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
239
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
240 gb_file_list = [f.strip() for f in args.input.split(",") if f.strip()]
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
241
2
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
242 # Connect to DB
0
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
243 MAX_RETRIES = 3
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
244 for attempt in range(1, MAX_RETRIES + 1):
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
245 try:
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
246 wait_for_db(db_uri)
2
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
247 break
0
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
248 except Exception as e:
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
249 if attempt == MAX_RETRIES:
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
250 print(f"Attempt {attempt} failed: Could not connect to database at {db_uri}.")
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
251 raise e
2
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
252 time.sleep(2)
0
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
253
2
dad2c4c3450a planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit db4ac861e1d03fcdfe94321d858839124e493930-dirty
tduigou
parents: 1
diff changeset
254 # Push annotations
0
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
255 push_gb_annotations(
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
256 gb_file_list,
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
257 sequence_column,
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
258 annotation_column,
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
259 db_uri,
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
260 table,
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
261 fragment_column,
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
262 args.output,
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
263 args.file_name_mapping
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
264 )
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
265
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
266
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
267 if __name__ == "__main__":
9f845ccb1a9f planemo upload for repository https://github.com/brsynth/galaxytools/tree/main/tools commit 3401816c949b538bd9c67e61cbe92badff6a4007-dirty
tduigou
parents:
diff changeset
268 main()