Mercurial > repos > dlalgroup > simtext_app

comparison README.md @ 0:34ed44f3f85c draft

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
"planemo upload for repository https://github.com/dlal-group/simtext commit fd3f5b7b0506fbc460f2a281f694cb57f1c90a3c-dirty"
author dlalgroup
date Thu, 24 Sep 2020 02:17:05 +0000
parents
children
comparison
equal deleted inserted replaced
-1:000000000000 0:34ed44f3f85c
1 # SimText
2
3 A text mining framework for interactive analysis and visualization of similarities among biomedical entities.
4
5 ## Brief overview of tools:
6
7 - pubmed_by_queries:
8
9 For each search query, PMIDs or abstracts from PubMed are saved.
10
11 - abstracts_by_pmids:
12
13 For all PMIDs in each row of a table the according abstracts are saved in additional columns.
14
15 - text_to_wordmatrix:
16
17 The most frequent words of text from each row are extracted and united in one large binary matrix.
18
19 - pmids_to_pubtator_matrix:
20
21 For PMIDs of each row, scientific words are extracted using PubTator annotations and subsequently united in one large binary matrix.
22
23 - simtext_app:
24
25 Shiny app with word clouds, dimension reduction plot, dendrogram of hierarchical clustering and table with words and their frequency among the search queries.
26
27 ## Requirements
28
29 - R (version > 4.0.0)
30
31 ## Installation
32
33 ```
34 $ mkdir -p <path>/simtext
35 $ cd <path>/simtext
36 $ git clone https://github.com/dlal-group/simtext
37 ```
38
39 ## pubmed_by_queries
40
41 This tool uses a set of search queries to download a defined number of abstracts or PMIDs for each search query from PubMed. PubMed's search rules and syntax apply. Users can obtain an API key from the Settings page of their NCBI account (to create an account, visit http://www.ncbi.nlm.nih.gov/account/).
42
43 Input:
44
45 Tab-delimited table with a list of search queries (biomedical entities of interest) in one column. The column header should start with "ID_" (e.g., "ID_gene" if search queries are genes).
46
47 Usage:
48 ```
49 $ Rscript pubmed_by_queries.R [-h] [-i INPUT] [-o OUTPUT] [-n NUMBER] [-a] [-k KEY] [--install_packages]
50 ```
51
52 Optional arguments:
53 ```
54 -h, --help show help message
55 -i INPUT, --input INPUT input file name. add path if file is not in working directory
56 -o OUTPUT, --output OUTPUT output file name [default "pubmed_by_queries_output"]
57 -n NUMBER, --number NUMBER number of PMIDs or abstracts to save per ID [default "5"]
58 -a, --abstract if abstracts instead of PMIDs should be retrieved use --abstracts
59 -k KEY, --key KEY if NCBI API key is available, add it to speed up the download of PubMed data
60 --install_packages if you want to auto install missing required packages
61 ```
62
63 Output:
64
65 A table with additional columns containing PMIDs or abstracts from PubMed.
66
67 ## abstracts_by_pmids
68
69 This tool retrieves abstracts for a matrix of PMIDs. The abstract text is saved in additional columns.
70
71 Input:
72
73 Tab-delimited table with rows representing biomedical entities and columns containing the corresponding PMIDs. The names of the PMID columns should start with “PMID_” (e.g., “PMID_1”, “PMID_2” etc.).
74
75 Usage:
76 ```
77 $ Rscript abstracts_by_pmid.R [-h] [-i INPUT] [-o OUTPUT]
78 ```
79
80 Optional arguments:
81 ```
82 -h, --help show help message
83 -i INPUT, --input INPUT input file name. add path if file is not in working directory
84 -o OUTPUT, --output OUTPUT output file name [default "abstracts_by_pmids_output"]
85 --install_packages if you want to auto install missing required packages
86 ```
87
88 Output:
89
90 A table with additional columns containing abstract texts.
91
92 ## text_to_wordmatrix
93
94 The tool extracts for each row the most frequent words from the text in columns starting with "ABSTRACT" or "TEXT. The extracted words from each row are united in one large binary matrix, with 0= word not frequently occurring in text of that row and 1= word frequently present in text of that row.
95
96 Input:
97
98 The output of ‘pubmed_by_queries’ or ‘abstracts_by_pmids’ tools, or a tab-delimited table with text in columns starting with "ABSTRACT" or "TEXT".
99
100 Usage:
101 ```
102 $ Rscript text_to_wordmatrix.R [-h] [-i INPUT] [-o OUTPUT] [-n NUMBER] [-r] [-l] [-w] [-s] [-p]
103 ```
104
105 Optional arguments:
106 ```
107 -h, --help show help message
108 -i INPUT, --input INPUT input file name. add path if file is not in working directory
109 -o OUTPUT, --output OUTPUT output file name. [default "text_to_wordmatrix_output"]
110 -n NUMBER, --number NUMBER number of most frequent words that should be extracted per row [default "50"]
111 -r, --remove_num remove any numbers in text
112 -l, --lower_case by default all characters are translated to lower case. otherwise use -l
113 -w, --remove_stopwords by default a set of english stopwords (e.g., 'the' or 'not') are removed. otherwise use -w
114 -s, --stemDoc apply Porter's stemming algorithm: collapsing words to a common root to aid comparison of vocabulary
115 -p, --plurals by default words in plural and singular are merged to the singular form. otherwise use -p
116 -- install_packages if you want to auto install missing required packages
117 ```
118
119 Output:
120
121 A binary matrix in that each column represents one of the extracted words.
122
123 ## pmids_to_pubtator_matrix
124
125 The tool uses all PMIDs per row and extracts "Gene", "Disease", "Mutation", "Chemical" and "Species" terms of the corresponding abstracts, using PubTator annotations. The user can choose from which categories terms should be extracted. The extracted terms are united in one large binary matrix, with 0= term not present in abstracts of that row and 1= term present in abstracts of that row. The user can decide if the scientific terms should be extracted and used as they are or if they should be grouped by their geneIDs/ meshIDs (several terms are often grouped into one ID). Also, by default all terms are extracted, otherwise the user can specify a number of most frequent words to extract per row.
126
127 Input:
128
129 Output of 'abstracts_by_pmids' tool, or tab-delimited table with columns containing PMIDs. The names of the PMID columns should start with "PMID", e.g. "PMID_1", "PMID_2" etc.
130
131 Usage:
132 ```
133 $ Rscript pmids_to_pubtator_matrix.R [-h] [-i INPUT] [-o OUTPUT] [-b BYID] [-n NUMBER][-c {Gene,Disease,Mutation,Chemical,Species} [{Gene,Disease,Mutation,Chemical,Species} ...]]
134 ```
135
136 Optional arguments:
137 ```
138 -h, --help show help message
139 -i INPUT, --input INPUT input file name. add path if file is not in workind directory
140 -o OUTPUT, --output OUTPUT output file name. [default "pmids_to_pubtator_matrix_output"]
141 -b, --byid if you want to find common gene IDs / mesh IDs instead of specific scientific terms.
142 -n NUMBER, --number NUMBER number of most frequent terms/IDs to extract. by default all terms/IDs are extracted.
143 -c [...], --categories [...] PubTator categories that should be considered [default "('Gene', 'Disease', 'Mutation','Chemical')"]
144 -- install_packages if you want to auto install missing required packages
145 ```
146
147 Output:
148
149 Binary matrix in that each column represents one of the extracted terms.
150
151 ## simtext_app
152
153 The tool enables the exploration of data generated by ‘text_to_wordmatrix’ or ‘pmids_to_pubtator_matrix’ tools in a Shiny local instance. The following features can be generated: 1) word clouds for each initial search query, 2) dimension reduction and hierarchical clustering of binary matrices, and 3) tables with words and their frequency in the search queries.
154
155 Input:
156
157 1) Input 1:
158 Tab-delimited table with
159 - A column with initial search queries starting with "ID_" (e.g., "ID_gene" if initial search queries were genes).
160 - Column(s) with grouping factor(s) to compare pre-existing categories of the initial search queries with the grouping based on text. The column names should start with "GROUPING_". If the column name is "GROUPING_disorder", "disorder" will be shown as a grouping variable in the app.
161 2) Input 2:
162 The output of ‘text_to_wordmatrix’ or ‘pmids_to_pubtator_matrix’ tools, or a binary matrix.
163
164 Usage:
165 ```
166 $ Rscript simtext_app.R [-h] [-i INPUT] [-m MATRIX] [-p PORT]
167 ```
168
169 Optional arguments:
170 ```
171 -h, --help show help message
172 -i INPUT, --input INPUT input file name. add path if file is not in working directory
173 -m MATRIX, --matrix MATRIX matrix file name. add path if file is not in working directory
174 -p PORT, --port PORT specify port, otherwise randomly selected
175 --host specify host
176 -- install_packages if you want to auto install missing required packages
177 ```
178
179 Output:
180
181 SimText app