a Փ?gW @sdZddlZddlZddlZddlZddlZddlZddlmZm Z m Z m Z ddl m Z ddlmZmZmZddlmZddlmZddlmZmZdd lmZmZmZmZmZmZm Z m!Z!m"Z"m#Z#m$Z$m%Z%m&Z&m'Z'm(Z(m)Z)dd l*m+Z+dd l,m-Z-dd l.m/Z/dd l0m1Z1ddl2m3Z3ddl4m5Z5ddl6m7Z7ddl8m9Z9ddl:m;Z;ddlm?Z?e@eAeBZCeCDejEe dddddddddddd d!d"d#ddd$d%d&d'ddgd(ZFd)gZGeFjHe eGd*d)gd)gd+d,gd-daId.aJe?ZKe+ZLe/eLeKZMe-eLZNe3eLZOe5eLe1e;eKeKZPe7eLZQe9eKZRe=eKZSeFTd/d0d1ZUeFTd/ed2d3d4d5ZVeFjWd6d7gd8d9d:ZXeFjYd;deeKjZfee[d?d@dAZ\eFjWdBdeeSj]fe[dDdEdFZ^eFj_dGdeeSj`feadIdJdKZbeFjWdLdgdMe)d>eeSj]eeKjZfece[e[dNdOdPZddQdRZeeFjYdSdgdTe(d>eeSj]eeKjZfee e[e[dUdVdWZfeFjYdXdgdYe'd>dZeeSj]eeKjZfeece[e[d[d\d]ZgeFjWd^dgd_e$d>d.eeSj]eeKjZfeae[e[d`dadbZhdddcZidddeZjeFjYdfd gdge#d>eeSj]eeKjZfee[e[dhdidjZkeFjWdkd gdldmdndeeKjZfeecece[dodpdqZleFjYdrd$gdse%d>eeKjZfee[d?dtduZmeFjWdvd$gdwdmdneeKjZfece[dxdydzZndS){a? Functional APIs provided by the server - /access/token - /access/oneoff - /access/revoke - /record/{uuid} - /graphql/query?mode=data/detail/facet/mri - /graphql/pagination?search= - /filter?sidebar= - /collection - /data/{action}/{filepath:path}?token= - /instance - /dicom/export/{identifier} N)DependsFastAPI HTTPExceptionstatus)CORSMiddleware) JSONResponseResponseStreamingResponse) repeat_every)find) Gen3Config iRODSConfig) ActionParamCollectionItemGraphQLPaginationItemGraphQLQueryItem IdentityItem InstanceItem ModeParamaccess_revoke_responsesaccess_token_responsescollection_responsesfilter_responsesinstance_responsesone_off_access_responsespagination_responsesquery_responsesrecord_responses) FilterEditor)FilterFormatter)FilterGenerator) FilterLogic)PaginationFormatter)PaginationLogic)QueryFormatter) QueryLogic) SearchLogic) Authenticator)ExternalServicez12 Labours Portala ## Access You will be able to: * **Create Gen3 access based on authority** * **Revoke Gen3 access** ## Gen3 You will be able to: * **Get Gen3 program/project** * **Get Gen3 node dictionary** * **Get Gen3 record(s) metadata** * **Use GraphQL query Gen3 metadata** * **Download Gen3 metadata file** ## iRODS You will be able to: * **Get iRODS root/sub-folder(s)/sub-file(s)** * **Download iRODS data file** ## Orthanc You will be able to: * **Get Orthanc dicom file instance ids** * **Download Orthanc dicom file** z!Auckland Bioengineering Institutez&https://www.auckland.ac.nz/en/abi.html)nameurlZGen3zEGen3 is a data platform for building data commons and data ecosystemszGen3 official websitezhttps://gen3.org/) descriptionr*)r)r+ externalDocsZiRODSz0iRODS is an open source data management softwareziRODS official websitezhttps://irods.org/ZOrthanczOOrthanc is a free and open-source, lightweight DICOM server for medical imagingzOrthanc official websitezhttps://www.orthanc-server.com/)titler+contact openapi_tags*T X-File-Name X-One-Off)Z allow_originsZallow_credentialsZ allow_methodsZ allow_headersZexpose_headersFstartupcstdattdS)z$ Create service connection. TN)EScheck_service_status CONNECTIONloggerinfor9r98/home/cmiss/Jenkins/workspace/12-Labours-API/app/main.pystart_ups r;iQ)secondsc Cs|datdrZz taWn0tyH}ztd|WYd}~n d}~00trdtdn tdt dkrxt dS)z7 Update filter and cleanup users periodically. Fgen3z)Invalid filter metadata %s has been used.Nz Default filter has been updated.z Failed to update default filter.) FILTER_GENERATEDr6FGZgenerate_public_filter Exceptionr7errorr8warningAZget_authorized_user_numberZcleanup_authorized_user)rBr9r9r:periodic_executions "   rE/ZRoot)tagscsdS)z Root z&This is the 12 Labours Portal backend.r9r9r9r9r:rootsrHz /access/tokenZAccessz'Create access token for authorized user)rGsummary responses)item connectioncsd|ddus|ddur&ttjdd|jdusD|jdusD|jdurRttjdddt|i}|S)z Return the access token. **Email** - email@gmail.com **Machine** - machine_id **Expiration** - expiration_time r=Nirods,Please check the service (Gen3/iRODS) status status_codedetail.Missing one or more fields in the request body access_token) rrHTTP_500_INTERNAL_SERVER_ERRORemailmachine expirationHTTP_400_BAD_REQUESTrDZgenerate_access_token)rKrLresultr9r9r: create_accesss rZz/access/oneoffz/Create one off access token for authorized user) authoritycsd|di}|S)zV Return the one off access token. Used for preview and download endpoint. one_off_tokenr9)r[rYr9r9r:create_one_off_accesssr]z/access/revokez!Revoke access for authorized user)revokecs&ddi}tj}|sd|d<t||dS)z+ Return revoke message if success. messagezSuccessfully revoke the accessz)Unable to remove default access authority)rPcontent)r HTTP_200_OKr)r^r`rPr9r9r: revoke_accesss rbz/record/{uuid}zGet gen3 record information)uuidr[rLc s|ddurttjdddd}g}|dD]:}||\}}|d|||d}d |vr.||d q.|s|ttjd dtd |d id |didS)zb Return record information in the Gen3 Data Commons. - **uuid**: uuid of the record. r=N&Please check the service (Gen3) statusrOcSs|d}|d|dfS)N-rr>)split)accessZ access_listr9r9r: handle_access6s z&get_gen3_record..handle_access access_scopejsonr_rz0Data does not exist or unable to access the datarecordr2r\r`headers)rrrTZ export_recordappendHTTP_404_NOT_FOUNDr) rcr[rLrhrecordsrgprogramZprojectrkr9r9r:get_gen3_record s*     rrcCs4i}t|dkr0t|}|tjt|}|S)zB Handler for generating private access and private filter r>)lencopydeepcopyremover ZGEN3_PUBLIC_ACCESSr@Zgenerate_private_filter)riZprivate_filterZprivate_accessr9r9r:_handle_private_filterMs     rwz/graphql/queryz'GraphQL query gen3 metadata information)rKmoder[rLcs|ddurttjdd|dvr8ttjd|dd|dkrrd |jvs\t|jd d krrttjd |d d|jdurttjd d|jdvrttjdd|jdkr|jdkrttjddt |t t |d|d|_ t|fdd}tt |d|didS)a` Return queries metadata records. The API uses GraphQL query language. **node** - experiment_query - dataset_description_query - manifest_query - case_query **filter** - {"field_name": ["field_value", ...], ...} **search** - string content, - only available in dataset_description/manifest/case nodes r=NrdrO)datarQZfacetZmrizThe query mode () is not provided in this APIry submitter_idr>zMode z9 only available when query one dataset in experiment nodez!Missing field in the request body)experiment_queryZdataset_description_queryZmanifest_queryZ case_queryzInvalid query node is usedr|z'Search does not provide in current nodericstdkrdSS)Nr>r)rsr9 query_resultr9r: handle_results z-get_gen3_graphql_query..handle_resultr2r\rl)rrrTHTTP_405_METHOD_NOT_ALLOWEDfilterrsrXnodesearchQFZset_query_modeset_private_filterrwrgQLZget_query_datarZprocess_data_output)rKrxr[rLrr9r~r:get_gen3_graphql_queryYsV           rz/graphql/paginationzDisplay datasetsr})rKrr[rLcs|ddurttjdd|r8|ddur8ttjddtt|d|d_t|}t\}}t ||}j durj durt |fdd d }t t|j|d d |d idS)a /graphql/pagination/?search= Return filtered/searched metadata records. The API uses GraphQL query language. - Default page = 1 - Default limit = 50 - Default filter = {} - Default search = "" - Default relation = "and" - Default access = gen3 public access repository - Default order = "published(asc)" **node** - experiment_pagination **filter(zero or more)** - {"gen3_node>gen3_field": [filter_name,...], ...} **search(parameter)**: - string content r=NrdrOrMz'Please check the service (iRODS) statusricsjd|dSNr{)rindex)dictrKr9r:z-get_gen3_graphql_pagination..)key)itemsZ numberPerPagetotalr2r\rl)rrrTPLrrwrgZprocess_pagination_itemZget_pagination_countZget_pagination_dataZascdescsortedrPFZreconstruct_data_structurelimit)rKrr[rLZis_public_access_filteredZ data_countZ match_pairrr9rr:get_gen3_graphql_paginations6"     rz/filterzGet filter information)sidebarr[rLcsd|ddurttjddd}|dkr>ts>|d7}t|qtt|d|r\t St S) z /filter/?sidebar= Return the support data for portal filters component. - **sidebar**: boolean content. r=NrdrOr r>ri) rrrTr?timesleepFFrrwZgenerate_sidebar_filter_formatZgenerate_filter_format)rr[rLretryr9r9r:get_gen3_filters   r)endpoint additionalcCsttd|d}i}|r@td|kr8td|kr8dS||d<td||d}td|}ttdd |}|st t j d d |td<|td<|S) NrFrrr{Zexperiment_filter)rrrgr=cSs|dSrr9)dr9r9r:r.rz&_handle_irods_access..zUnable to access the datarO) listrrf IRODS_REQUESTrr4getZprocess_graphql_querymaprrHTTP_401_UNAUTHORIZED)rpathriZdatasetZfilter_Z query_itemr accessibler9r9r:_handle_irods_accesss0  r /collectionzGet folder information)rKr[rLc s|ddus|ddur&ttjddtdjsBttjddtdj|d fd d }z8|dj t j j}||j ||j d }|WSty}zttjd d|WYd}~n d}~00dS)zy Return all collections from the required folder. Root folder will be returned if no item or "/" is passed. r=NrMrNrOz(/(.)*)+zInvalid path format is usedrric sHg}|D]:}jdkr"|jvr"q||jttjd|jdq|S)NrFr})r)r)rr)rnresubr IRODS_ROOT_PATH)ry collectionZelerrKr9r:handle_collectionYsz/get_irods_collection..handle_collection)Zfoldersfiles#Data not found in the provided path)rrrTrmatchrrXr collectionsrr rZsubcollections data_objectsrAro)rKr[rLrZcollrYrBr9rr:get_irods_collection:s:   rz/data/{action}/{filepath:path}zDownload irods filez$Successfully return a file with data)rGrIresponse_description)actionfilepathtokenrLc sd|ddus|ddur*ttjdddvrHttjdd dt|}td ||z&|djt j d |j Wn4t y}zttj d d|WYd}~n d}~00fd d}fdd}fdd}t|||dS)a$ Used to preview most types of data files in iRODS (.xlsx and .csv not supported yet). OR Return a specific download file from iRODS or a preview of most types data. - **action**: Action should be either preview or download. - **filepath**: Required iRODS file path. i@r=NrMrNrO)Zpreviewdownloadz The action (rzz/data/rFrcs d}dkrdd}|S)Nrzattachment;filename=)r1zContent-Dispositionr9)header)rfilenamer9r: handle_headers z*get_irods_data_file..handle_headercstdS)Nr) mimetypes guess_typer9)rr9r:handle_mimetypesz,get_irods_data_file..handle_mimetypec3sNd0}|}|r,|V|}qWdn1s@0YdS)Nr)openread)Z file_likechunk) chunk_sizefiler9r: iterate_files   z)get_irods_data_file..iterate_file) media_typerm)rrrTrrDZhandle_get_one_off_authorityrrrr rr)rAror ) rrrrLrirBrrrr9)rrrrr:get_irods_data_filevs>      rz /instancezGet instance idscs|ddurttjddjdus.jdur.cs |jjkSr)rseriesrrr9r:rr)rZ study_filterZ series_filter+Resource is not found in the orthanc server) rrrTrrrXr roZstudiesZ instancesrnZid_)rKrLZpatientsrYZpatientrrinstancer9rr:get_orthanc_instances6      rz/dicom/export/{identifier}zExport dicom file) identifierrLc s~|ddurttjddz*|d|}t|}t|ddWStyx}zttj dd|WYd}~n d}~00dS)zf Export a specific dicom file from Orthanc server - **identifier**: dicom instance uuid. rNrrOzapplication/dicom)rr) rrrTZget_instances_id_fileioBytesIOrgetvaluerAro)rrLZ instance_fileZ bytes_filerBr9r9r:get_orthanc_dicom_files   r)o__doc__rtrloggingrrrfastapirrrrZfastapi.middleware.corsrfastapi.responsesrrr Zfastapi_utils.tasksr Z pyorthancr Z app.configr r Zapp.data_schemarrrrrrrrrrrrrrrrZ!app.function.filter.filter_editorrZ$app.function.filter.filter_formatterrZ$app.function.filter.filter_generatorr Z app.function.filter.filter_logicr!Z,app.function.pagination.pagination_formatterr"Z(app.function.pagination.pagination_logicr#Z"app.function.query.query_formatterr$Zapp.function.query.query_logicr%Z app.function.search.search_logicr&Zmiddleware.authr'Zservices.external_servicer( basicConfig getLogger__name__r7setLevelINFOappZoriginsadd_middlewarer6r?r4ZFEr@rrrrrrDon_eventr;rErrHpostr5rrZZhandle_get_authorityr]deleteZhandle_revoke_authorityboolrbstrrrrwrrrrrrrrrr9r9r9r:s   H             !  D       '  N<"6 F&