a áYšg!7ã @s^ ddlZddlZddlmZddlmZmZmZddlm Z m Z ddl m Z m Z mZmZddlmZmZddlmZddlmZmZdd lmZddlZddlZddlZddlZddlZddlZddlZdd l m!Z!ddl"Z"ddl#Z#ddl$Z$ddl%Z%ddl&Z&dd l'm(Z(dd l)m*Z*dd l+m,Z,ddl-m.Z.ddl/m0Z0ddl1m2Z2ddl3m3Z3m4Z4ddl5m6Z6m7Z7m8Z8m9Z9ddl:m;Z;ddlm?Z?ddl@m?ZAddlBmCZCddlDmEZFddlGmHZHddlImJZJddlKmLZLddlMmNZNmOZOmPZPmQZQmRZRmSZSmTZTmUZUmVZVmWZWmXZXmYZYmZZZm[Z[m\Z\ddl]m^Z^m_Z_m`Z`maZambZbmcZcddl&mdZdddlemfZfddlgmhZhdd limjZjmkZkmlZldd!lmmnZnmoZompZpmqZqmrZrmsZsmtZtdd"lumvZvdd#lwmxZxmyZydd$lzm{Z|dd%lzm}Z~dd&lmnZ€mZ‚e# ƒ¡e6e„ƒZ…ehj† ‡¡Zˆe…j‰ Še‹e#eˆe#jŒƒ¡ehje…jŽd'<eLe…d(d)d*œd+Ze;e…ƒe=e…ƒZe^ƒZ‘da’ej“d,ehj”ehj•d-d.Z–edƒZ—zejehj˜ƒZ™Wnešy†dZ™Yn0zekehj˜ƒZ›Wnešy®dZ›Yn0zelehj˜ƒZœWnešyÖdZœYn0Gd/d0„d0eƒZže… Ÿd1¡d2d3„ƒZ e…j¡d4d5„ƒZ¢e…j¡d6d7„ƒZ£e*ƒZ¤e*ƒZ¥e*ƒZ¦e*ƒZ§e*ƒZ¨e§j©sNe# ªd8¡e§ «¡ehjd9kr’e*ƒZ¬eƒZ­e­ ®¡e¬ «¡e¬j¯e­j®d:d;d;dehjd?krÖehj°d@krÖe¨j©sÄe# ªdA¡e¨ «¡e¨j¯e d:dBdCdDiZ±e…j¡dEdF„ƒZ²iZ³eƒZ´e ƒZµeƒZ¶e…j¡dGdH„ƒZ·eƒZ¸e…j¡dIdJ„ƒZ¹e¦j¯e¹dKdLdMe¤j¯e²dKdLdMe¥j¯e·dKdNdOe,e.ƒe0dLdPgƒZºe§ ¯dQdR„eº¡dSdT„Z»e ¼e»¡e… ½dU¡dVdW„ƒZ¾e…j½dXdYgdZd[d\„ƒZ¿d]d^„ZÀe… ½d_¡d`ehjÁfdadb„ƒZÂe… ½dc¡ddde„ƒZÃe… ½df¡ehjÁfdgdh„ƒZÄe… ½di¡ehjÁfdjdk„ƒZÅdldm„ZÆe… ½dn¡dodp„ƒZÇdqdr„ZÈe… ½ds¡ehjÁfdtdu„ƒZÉe… ½dv¡dwdx„ƒZÊe… ½dy¡e… ½dz¡d{d|„ƒƒZËe… ½d}¡d~d„ƒZÌe… ½d€¡dd‚„ƒZÍe… ½dƒ¡e… ½d„¡d…d†„ƒƒZÎe… ½d‡¡e… ½dˆ¡d‰dŠ„ƒƒZÏe… ½d‹¡e… ½dŒ¡ddŽ„ƒƒZÐe… ½d¡dd‘„ƒZÑe… ½d’¡d“d”„ƒZÒe… ½d•¡d–d—„ƒZÓe… ½d˜¡d™dš„ƒZÔe… ½d›¡ehjÁfdœd„ƒZÕe… ½dž¡dŸd „ƒZÖd¡d¢„Z×e…j½d£d¤d¥dd¦œd§e… ½d¨¡dAd©dª„ƒƒZØe…j½d«d¬d¤id§e… ½d­¡d®d¯„ƒƒZÙe… ½d°¡d±d²„ƒZÚd³d´„ZÛdµd¶„ZÜd·d¸„ZÝe… ½d¹¡dºd»„ƒZÞe… ½d¼¡d½d¾„ƒZße… ½d¿¡dÀdÁ„ƒZàe… ½d¡dÃdÄ„ƒZáe… ½dÅ¡dÆdÇ„ƒZâe…j½dÈdÉgdZdÊdË„ƒZãe…j½dÌdÉgdZdÍd΄ƒZäe…j½dÏdÉgdZejåd)dÐdÑdÒ„ƒƒZæe…j½dÓdÉgdZdÔdÕ„ƒZçe…j½dÖdÉgdZd×dØ„ƒZèe…j½dÙdÉgdZdÚdÛ„ƒZée…j½dÜdÉgdZdÝdÞ„ƒZêe…j½dßdÉgdZdàdᄃZëe…j½dâdÉgdZdBdäd儃Zìe…j½dædÉgdZdçd脃Zíe…j½dédÉgdZdêd넃Zîe…j½dìdÉgdZdídZïe…j½dïdÉgdZdðdñ„ƒZðdòdó„Zñdôdõ„Zòdöd÷„Zóe…j½dødYgdZdùdú„ƒZôe…j½dûdYgdZdüdý„ƒZõe…j½dþdYgdZdÿd„ƒZöe…j½ddYgdZdd„ƒZ÷e…j½ddYgdZdd„ƒZøe…j½ddÉgdZdd „ƒZùe…j½d dYgdZd d „ƒZúd d„Zûdd„Züe…j½ddYgdZdd„ƒZýe…j½ddYgdZdd„ƒZþe…j½ddYgdZdd„ƒZÿe…j½ddÉgdZdd„ƒZe… ½d¡dd„ƒZe… ½d ¡d!d"„ƒZe… ½d#¡d$d%„ƒZe…j½d&dYgdZd'd(„ƒZe…j½d)dYgdZd*d+„ƒZ{e…j½d,dYgdZd-d.„ƒZ}e…j½d/dYgdZd0d1„ƒZe… ½d2¡d3d4„ƒZe…j½d5dÉgdZd6d7„ƒZe…j½d8dÉgdZd9d:„ƒZe…j½d;dYgdZd<d=„ƒZ e…j½d>dÉgdZd?d@„ƒZ dS(CéN)Úget_download_count)Úinit_cf_cda_clientÚget_funded_projects_countÚget_featured_datasets)Úupdate_all_events_sort_orderÚupdate_event_sort_order)Úget_dataset_countÚinit_algolia_clientÚget_all_dataset_idsÚget_associated_datasets)Úinit_ga_reportingÚget_ga_1year_sessions)Ú MonthlyStats)Úset_featured_dataset_idÚ#get_featured_dataset_id_table_state)ÚOSparcServices)Ú ApiException)Úurlparse)ÚBackgroundScheduler)Ú OrTrigger)Ú DateTrigger)ÚIntervalTrigger)Ú ClientError)ÚdatetimeÚ timedelta)ÚFlaskÚabortÚjsonifyÚrequest)ÚCORS)Ú Marshmallow)Ú Pennsieve)Ú new_client)ÚUnauthorizedException)ÚImage)Ú HTTPBasicAuth)ÚCache)Úcreate_doi_queryÚcreate_filter_requestÚcreate_facet_queryÚcreate_doi_aggregateÚcreate_title_queryÚcreate_identifier_queryÚ!create_pennsieve_identifier_queryÚcreate_field_queryÚcreate_request_body_for_curiesÚcreate_onto_term_queryÚcreate_multiple_doi_queryÚ create_multiple_discoverId_queryÚcreate_anatomy_queryÚget_body_scaffold_dataset_idÚcreate_multiple_mimetype_query)Ú EmailSenderÚfeedback_emailÚgeneral_interest_emailÚissue_reporting_emailÚ#creation_request_confirmation_emailÚservice_interest_email)ÚLock)Ú ElementTree)ÚConfig)ÚMapTableÚ ScaffoldTableÚFeaturedDatasetIdSelectorTable)Úprocess_resultsÚprocess_get_first_scaffold_infoÚreform_aggregation_resultsÚreform_curies_resultsÚreform_dataset_resultsÚreform_related_termsÚreform_anatomy_results)ÚContactRequestSchema)Úimg_to_base64_strÚget_path_from_mangled_list)Ústart_simulation)Úcheck_simulation)rBÚprocess_resultZENVZ SimpleCacheé,)Z CACHE_TYPEZCACHE_DEFAULT_TIMEOUT)ÚconfigÚs3z us-east-1)Zaws_access_key_idZaws_secret_access_keyZ region_namec@s`eZdZdZe ¡edƒZdZe dd„ƒZ dd„Z e dd „ƒZ d d „Z e d d „ƒZdd„ZdS)Ú BiolucidaÚi?BFcCs |t_dS©N)rRÚ_token©Úvalue©rXú3/home/cmiss/Jenkins/workspace/SPARC-API/app/main.pyÚ set_tokennszBiolucida.set_tokencCs|jSrT)rU©ÚselfrXrXrYÚtokenrszBiolucida.tokencCs |t_dSrT)rRÚ _expiry_daterVrXrXrYÚset_expiry_dateuszBiolucida.set_expiry_datecCs|jSrT)r^r[rXrXrYÚ expiry_dateyszBiolucida.expiry_datecCs |t_dSrT)rRÚ_pending_authenticationrVrXrXrYÚset_pending_authentication|sz$Biolucida.set_pending_authenticationcCs|jSrT)rar[rXrXrYÚpending_authentication€sz Biolucida.pending_authenticationN)Ú__name__Ú __module__Ú __qualname__rUrÚnowrr^raÚ staticmethodrZr]r_r`rbrcrXrXrXrYrRis   rRé”cCstt|ƒddfS)N)Úerrorri)rÚstr)ÚerXrXrYÚresource_not_found„srmc CsÈzttjtjdtjdaWn¨tjjyV}z t   d¡t   |¡WYd}~nvd}~0t yŒ}z t   d¡t   |¡WYd}~n@d}~0t yÂ}z t   d¡t   |¡WYd}~n d}~00dS)NF)Z api_tokenÚ api_secretZ env_overrideÚhostz#Unable to connect to Pennsieve hostz&Unable to authorise with Pennsieve Apiz Unknown Error) r!r>ÚPENNSIEVE_API_TOKENÚPENNSIEVE_API_SECRETÚPENNSIEVE_API_HOSTÚpsÚrequestsÚ exceptionsÚ HTTPErrorÚloggingrjÚPSUnauthorizedExceptionÚ Exception©ÚerrrXrXrYÚconnect_to_pennsieve‰s ü    r|c CsTzttjtjtjddaWn4tyN}zt d|›¡WYd}~n d}~00dS)N)Úapi_keyrnZapi_hostZ api2_hostz'Error connecting to pennsieve 2 agent: ) r"r>rprqrrÚps2ryrwrjrzrXrXrYÚconnect_to_pennsieve2srz6Starting scheduler for featured dataset id acquisitionÚ productionZcronÚ*Ú1Ú2)ÚyearÚmonthÚdayÚhourÚminuteÚsecondZ developmentZFALSEz8Starting scheduler for updating contentful event entrieséz US/Eastern)r‡Útimezonec Cs¤t d¡zLtjtj›dd}|jrTd|j dd¡vrT| ¡}t |dƒ}|t d<Wn0t y†}zt  d |¡WYd}~n d}~00t js t d ¡t  ¡dS) NzGetting oSPARC viewersz/viewers©Úurlúapplication/jsonú content-typerSÚdataÚ file_viewersz!Could not retreive oSPARC viewersz1Starting scheduler for oSPARC viewers acquisition)rwÚinfortÚgetr>ÚOSPARC_API_HOSTÚokÚheadersÚjsonÚbuild_filetypes_tableÚ osparc_dataryrjÚviewers_schedulerÚrunningÚstart)ÚreqZviewersÚtablerlrXrXrYÚget_osparc_file_viewersÇs   " rŸcCspt d¡trttƒ}|td<tr2ttƒ}|td<trFttƒ}|td<t ƒ}|td<t j slt d¡t   ¡dS)NzGathering metrics dataZ1year_sessions_countZ dataset_countZfunded_projects_countZ1year_download_countz*Starting scheduler for metrics acquisition) rwr’Úgoogle_analyticsr Ú usage_metricsÚalgoliarÚ contentfulrrÚmetrics_schedulerr›rœ)Z ga_responseZalgolia_responseZ cf_responseZ ps_responserXrXrYÚ get_metricsÞs  r¥c Csnt d¡z.tjtj›dd}| ¡}t |d¡Wn0t yh}zt  d|¡WYd}~n d}~00dS)NzFetching oSPARC servicesz /servicesrŒrz%Request to get oSPARC services failed) rwr’rtr“r>r”r—Úosparc_servicesZ set_servicesryrj)rZ services_resprlrXrXrYÚ get_services÷s r§Úintervalé)ÚfuncÚtriggerÚdaysé)rªr«Úhours)r®cCsttƒSrT)rÚfeaturedDatasetIdSelectorTablerXrXrXrYÚ ór°cCs|t d¡tjrt ¡t d¡tjr0t ¡t d¡tjrHt ¡t d¡tjr`t ¡t d¡tjrxt ¡dS)Nz1Stopping scheduler for oSPARC viewers acquisitionz*Stopping scheduler for metrics acquisitionz2Stopping scheduler for updating contentful entriesz3Stopping scheduler for updating featured dataset idz&Stopping scheduler for oSPARC services) rwr’ršr›Úshutdownr¤Ú)update_contentful_event_entries_schedulerÚfeatured_dataset_id_schedulerÚservices_schedulerrXrXrXrYÚshutdown_schedulerss     r¶z/healthcCst ddi¡S)NÚstatusZhealthy)r—ÚdumpsrXrXrXrYÚhealth$sr¹z/contactÚPOST)Úmethodsc Csht tj¡}tƒ |¡}|d}|d}|d}t |||¡t t j |dt   d|i¡¡t  ddi¡S)NÚnameÚemailÚmessagezFeedback submissionr·Úsent)r—ÚloadsrrrIÚloadÚ email_senderZ send_emailÚsendgrid_emailr>Ú SES_SENDERr7Ú substituter¸)rZcontact_requestr¼r½r¾rXrXrYÚcontact)s  rÆcCstjd||d|dœ|d}|S)NÚ get_objectÚ requester)ÚBucketÚKeyÚ RequestPayerZResponseContentType)ZParamsZ ExpiresIn)rQZgenerate_presigned_url)Ú s3BucketNameÚkeyÚ content_typeÚ expirationÚresponserXrXrYÚcreate_s3_presigned_url7s  ýrÑz /downloadicCs6tj d¡}tj d|¡}tj dd¡}t||||ƒS)NrÍrÌZ contentTypezapplication/octet-stream)rÚargsr“rÑ)rÏÚ bucket_namerÍrÌrÎrXrXrYÚcreate_presigned_urlBs rÔz/thumbnail/neurolucidac Cstj}d|vsd|vsd|vr*tdddStj›d}zPtj||dd }| ¡|jd krz|j  d d ¡d krzt   |j ¡WStddƒWn|tj jy¨tdddYStj jyÆtddƒYStj jy}z tddt|ƒ›ƒWYd}~Sd}~00dS)NÚversionZ datasetIdÚpathéúQuery arguments are not valid.©Ú descriptionz /thumbnailé)ÚparamsÚtimeoutéÈú Content-TypeÚunknownz image/pngzFailed to retrieve thumbnail.z0Unable to make a connection to NEUROLUCIDA_HOST.iøz&Request to NEUROLUCIDA_HOST timed out.éöz)Error while requesting NEUROLUCIDA_HOST: )rrÒrr>ZNEUROLUCIDA_HOSTrtr“Úraise_for_statusÚ status_coder–Úbase64Ú b64encodeÚcontentruÚConnectionErrorZTimeoutÚRequestExceptionrk)Ú query_argsrrÐrlrXrXrYÚthumbnail_from_neurolucida_fileKs"   rêz/thumbnail/segmentationc Cstj}d|vrtdddS| d|¡}|d}d}d}d}d}d }|} |rP|sTz"tj||d |›d | ›d d } Wnvtyè} z^| jdddkr´tdd|›ddWYd} ~ Stdd|›ddWYd} ~ SWYd} ~ n d} ~ 00| d ¡  d¡}d|v}d|v}|r"|s"| |7} n||7}| |7} t |ƒ|krFtdd|›ddSqF|durrtdd|›ddS||  d¡d…} | d|   d¡…d} t   | ¡}|j}d}|D]}||jdd…7}q²t |¡}t dt|dƒt|dƒf|¡}t|ƒ}|S)z° Extract a thumbnail from a mbf xml file. First phase is to find the thumbnail element in the xml document. Second phase is to convert the xml to a base64 png. rÖr×rØrÙrÌNFrièzbytes=ú-rÈ)rÉrÊZRangerËÚErrorÚCodeÚ NoSuchKeyriúCould not find file: 'ú'úUnknown error for file: 'ÚBodyzUTF-8z z#Could not find thumbnail in file: 'rSrŠZRGBZrowsÚcols)rrÒrr“rQrÇrrÐÚreadÚdecodeÚlenÚfindr=Ú fromstringÚattribÚtextÚbytesÚfromhexr$Ú frombytesÚintrJ)rÓrérÌrÖÚresourceZstart_tag_foundZ end_tag_foundZ start_byteÚoffsetZend_byterÐÚexZstart_thumbnail_elementZ thumbnail_xmlÚxmlZ size_infoZim_dataÚchildZ byte_im_dataZimZ base64_formrXrXrYÚextract_thumbnail_from_xml_filebsZ   ü  6      "rz/exists/cCsftj}| d|¡}ztj||dd}Wnty@ddiYS0| dd¡}|dkr^ddiSddiS) NrÌrÈ©rÉrÊrËÚexistsÚfalseÚ ContentLengthrÚtrue)rrÒr“rQÚ head_objectr)rÖrÓrérÌÚ head_responseÚcontent_lengthrXrXrYÚ url_exists¡s ý   r cCs0| dd¡}|d| d¡…}t |¡}| ¡S)Nzhttps://api.pennsieve.io/z"https://api.pennsieve.io/discover/ú/)ÚreplaceÚrfindrtr“r—)ÚuriÚrrXrXrYÚfetch_discover_file_information¸s  rz/s3-resource/discover_pathc Cs†tj d¡}z6t|ƒ}d|vr@|ddkr@|dd}|dWSWn0tyr}zt d|¡WYd}~n d}~00tdd |›d S) NrZ totalCountr©ÚfilesrrÖzFailed to retrieve uri {uri}rizFailed to retrieve uri rÙ)rrÒr“rryrwrjr)rZ json_responseÚ file_inforrXrXrYÚget_discover_pathÁs  "rc CsäzDtj||dd}| dtj¡}|rB|tjksBtdd|›dWSWn–tjjyÚ}zz|j ddd krzdWYd}~S|j ddd krœdWYd}~St|j dd|j ddƒWYd}~SWYd}~nd}~00dSdS)NrÈrrizFile too big to download: rÙrìríZ404riú.Provided path was not found on the s3 resourceZ403é“úEThere is a permission issue when accessing the file at specified pathÚMessage)rÞÚOK)rir)rr) rQr r“r>ZDIRECT_DOWNLOAD_LIMITrÚbotocorerurrÐ)rÖrÓr r r{rXrXrYÚs3_header_checkÎs ý@rz/s3-resource/c Csàtj}| d|¡}|}t||ƒ}|ddks8|ddkr¢t|ƒ}||krTtdddt||ƒ}|ddkrp|}n2|ddkrŠtdddn|ddkr¢tdddtj||d d }tj d ¡}|d  ¡} |durÜt   | ¡S| S) NrÌrrirrrÙrÞrrÈrZ encodeBase64rò) rrÒr“rrKrrQrÇrôrärå) rÖrÓrérÌZs3_pathrÐZs3_path_modifiedZ response2Ú encode_base64rÿrXrXrYÚdirect_download_urlçs2        ý   rz /scicrunch-dataset//c Csˆ| dd¡d|}t|ƒ}z&tjtj›dtj›|d}| ¡WStjj y‚}z$t   |¡t  d|i¡WYd}~Sd}~00dS)NzDOI:rSrú/_search?api_key=©r—rj) rr'rtÚpostr>ÚSCI_CRUNCH_HOSTÚKNOWLEDGEBASE_KEYr—rurvrwrjr¸)Zdoi1Zdoi2ÚdoirrÐr{rXrXrYÚsci_doisþ  r&z /pubmed/z/pubmed//c Cs`zt d|›d¡}|jWStjjyZ}z$t |¡t d|i¡WYd}~Sd}~00dS)Nz https://pubmed.ncbi.nlm.nih.gov/rrj) rtr“rúrurvrwrjr—r¸)Úid_rÐr{rXrXrYÚpubmeds  r(z/scicrunch-query-string/c Cs®tj d¡}tj d¡}tj d¡}tj d¡}t||||ƒ}z*tjtj›dtj ›|d}t |  ¡ƒWStj j y¨}z$t |¡t  d|i¡WYd}~Sd}~00dS)NÚfieldÚcurieÚsizeÚfromr r!rj)rrÒÚgetlistr“r.rtr"r>r#r$rBr—rurvrwrjr¸)Úfieldsr*r+Zfrom_rrÐr{rXrXrYÚ sci_organ+s    þ r/z/dataset_info/using_doicCs<tj d¡}tj d¡}t|ƒ}|dur4tt|ƒƒSt|ƒS)Nr%Z raw_response)rrÒr“r'rFÚdataset_search)r%ÚrawÚqueryrXrXrYÚget_dataset_info_doi>s    r3z!/dataset_info/using_multiple_doisz"/dataset_info/using_multiple_dois/cCs tj d¡}t|ƒ}tt|ƒƒS)NÚdois)rrÒr-r1rBr0)r4r2rXrXrYÚget_dataset_info_doisIs r5z./multiple_dataset_info/using_multiple_mimetypez//multiple_dataset_info/using_multiple_mimetype/cCs tj d¡}t|ƒ}tt|ƒƒS)NÚq)rrÒr-r5rBr0)r6r2rXrXrYÚget_file_info_from_mimetypeQs r7z(/dataset_info/using_multiple_discoverIdsz)/dataset_info/using_multiple_discoverIds/cCs tj d¡}t|ƒ}tt|ƒƒS)NÚ discoverIds)rrÒr-r2rBr0)r8r2rXrXrYÚget_dataset_info_discoverIds[s r9z/dataset_info/using_titlecCs tj d¡}t|ƒ}tt|ƒƒS)NÚtitle)rrÒr“r+rFr0)r:r2rXrXrYÚget_dataset_info_titleds r;z%/dataset_info/using_object_identifiercCs tj d¡}t|ƒ}tt|ƒƒS©NÚ identifier)rrÒr“r,rFr0©r=r2rXrXrYÚ"get_dataset_info_object_identifierls r?z/dataset_info/anatomycCs6tj dd¡}|dkr"tdddSt|ƒ}tt|ƒƒS)Nr=éÿÿÿÿriz Identifier for API call not set.rÙ)rrÒr“rr3rHr0r>rXrXrYÚget_dataset_info_anatomyts  rAz(/dataset_info/using_pennsieve_identifiercCs tj d¡}t|ƒ}tt|ƒƒSr<)rrÒr“r-rFr0r>rXrXrYÚ%get_dataset_info_pennsieve_identifiers rBz/segmentation_info/c CsZtj}d|vrtdddS| d|¡}| d¡}z6t||ƒ}|ddkrTtdddtj||d d }WnvtyÜ}z^|jd d d kr¨tdd|›ddWYd}~Stdd|›ddWYd}~SWYd}~n d}~00|d  ¡}t   |¡}|  d¡}i} |dur|j | d<ndddddœ| d<|  d¡} | durJ| j | d<n ddi| d<| S)NÚ dataset_pathr×z,Query arguments must include 'dataset_path'.rÙrÌrrirrÈrrìrírîrïrðrñròz./{*}sparcdata/{*}subjectÚsubjectrS)ZageZsexÚspeciesZ subjectidz./{*}sparcdata/{*}atlasZatlasZorgan)rrÒrr“rrQrÇrrÐrôr=rør÷rù) rÓrérÌrCrÐrrÿrZsubject_elementr’Z atlas_elementrXrXrYÚget_segmentation_info_from_file‡s<      ý  6         rFz/current_doi_listcCsBtƒ}tt|ƒƒ}g}|ddD]}| |dd¡q"d|iS)Nr%ÚbucketsrÍr*Úresults)r*rDr0Úappend)r2rHZ doi_resultsÚresultrXrXrYÚ get_all_doi´s  rKc Cstz0|}dtji}tjtj›d||d}| ¡WStjjyn}z"t  |¡t d|iƒWYd}~Sd}~00dS)Nr}ú/_search)r—rÜrj) r>r$rtr"r#r—rurvrwrjr)r2ÚpayloadrÜrÐr{rXrXrYr0¿sÿÿ  r0z/search/rSé )r2Úlimitrœ)Údefaultsz/search/c CsÒzŒtj d¡durtj d¡}tj d¡dur:tj d¡}tj d¡durVtj d¡}t tj›d|›d|›d|›dtj› ¡}t| ¡ƒWStj j yÌ}z$t   |¡t  d|i¡WYd}~Sd}~00dS) NrOr2rœz /_search?q=z&size=z&from=z &api_key=rj)rrÒr“rtr>r#r$rBr—rurvrwrjr¸)r2rOrœrÐr{rXrXrYÚ kb_searchÑs   * rQz/filter-search/r2z/filter-search//c CsÞtj d¡}tj d¡}tj d¡}tj d¡}t|||||ƒ}z,tjtj›dtj ›|d}t |  ¡ƒ}Wnltj j y´}z,t |¡tt|ƒddœƒd fWYd}~Sd}~0t jyØtd d d œƒd fYS0|S) NÚtermZfacetr+rœr r!zr#r$rBr—rurvrwrjrrkrU) r2ÚtermsZfacetsr+rœrrÐrHr{rXrXrYÚ filter_searchås*    þ *ÿÿ rXz/get-facets/c Csòt|ƒ\}}g}||D]¬}||d|›dd<z2tjtj›dtj›|d}| ¡}| |¡WqtjyŽt dddœƒd fYSt yÂ}zt   d |›|¡WYd}~qd}~00qg}|D]} || d|›d 7}qÎt |ƒS) NZ aggregationsrWr)r r!rTrUrVráz$Could not search SciCrunch for path rG) r)rtr"r>r#r$r—rIrUrryrwrj) Útype_Ztype_maprrHrÖrÐZ json_resultrrWrJrXrXrYÚ get_facetss.  þÿÿ *rZcCs&d|vr"t | d¡¡}|j|d<dS)NZreadmeZmarkdown)rtr“rú)ÚrespZmark_reqrXrXrYÚinject_markdownsr\c CsB| d¡}| d¡}|dus$|dur(dSt|ƒ}|j}ztj|d |¡dd}Wn|tyÌtjdk}|rvt   d¡ztj|d |¡dd}Wn6tyÆ}z|r®t   |¡WYd}~YdSd}~00Yn0|d   ¡}zt  |¡} Wn2ty}zt   |¡WYd}~dSd}~00|  d ¡|  d ¡|  d ¡d œ|d<dS)NÚidrz{}/files/template.jsonrÈrZTRUEzXRequired file template.json was not found under /files folder, trying under /packages...z{}/packages/template.jsonròÚuuidr¼rÚ)r^r¼rÚZstudy)r“rÚnetlocrQrÇÚformatrr>ÚSPARC_API_DEBUGGINGrwÚwarningrjrôr—rÀÚ ValueError) r[r'rZ parsed_uriÚbucketrÐÚ debuggingrlÚtemplateZ template_jsonrXrXrYÚinject_template_data$sL  ý   ÿý     ýrgcCsFi}|D]8}|d ¡}|d=| |d¡s2g||<|| |¡q|S)NÚ file_typeF)Úlowerr“rI)Zosparc_viewersržZviewerZfiletyperXrXrYr˜Ts  r˜z/sim/dataset/c Cs’tjdkrŽz@t d tj|¡¡}|jrH| ¡}t |ƒt |ƒt |ƒWSWn6t y€}zt  d|›|¡WYd}~n d}~00tdddSdS)NÚGETz{}/datasets/{}úCould not fetch SIM dataset riúResource not foundrÙ©rÚmethodrtr“r`r>ÚDISCOVER_API_HOSTr•r—r\rgrryrwrjr)r'rÚ json_datarrXrXrYÚ sim_dataset^s (rqz&/sim/dataset//versions/c Csštjdkr–zBt d tj||¡¡}|jrJ| ¡}t |ƒt |ƒt |ƒWSWn<t yˆ}z$t  d|›d|›|¡WYd}~n d}~00tdddSdS)Nrjz{}/datasets/{}/versions/{}rkz version rirlrÙrm)r'Zversion_rrprrXrXrYÚsim_dataset_versionsls .rrz/get_osparc_datacCsttƒSrT)rr™rXrXrXrYÚget_osparc_data|srsz /sim/servicecCsTtjdkrPtj d¡}tjjddtd}tjjddtd}t |||¡}t|ƒSdS)NrjÚsearchrOrÛ)ÚdefaultÚtypeÚskipr)rrnrÒr“rþr¦Zsearch_servicesr)rtrOrwrHrXrXrYÚ osparc_searchs   rxz /sim/filecCs tjdkrtj}td|iƒSdS)Nrjr‘)rrnr¦Zfile_extensionsr)Ú extensionsrXrXrYÚosparc_extensions‹s  ÿrzz/project/rjcCs4t|ƒ}t|dƒdkr$t|dƒStddddS)NÚhitsrrirlrÙ)r rörr)Z project_idÚdatasetsrXrXrYÚdatasets_by_project_id”s r}z"/get_featured_datasets_identifierscCs dtƒiS)NÚ identifiers)rrXrXrXrYÚ!get_featured_datasets_identifiersžsrz/get_featured_dataset)rÝc Cs¢ttƒd}|dkrd}zBt d tj|¡¡ ¡}|dgkrVt d tjd¡¡ ¡}|WSty}zt   d|›|¡WYd}~n d}~00t ddd dS) NÚfeatured_dataset_idr@é z{}/datasets?ids={}r|zCould not get featured dataset riz,An error occured while fetching the resourcerÙ) rr¯rtr“r`r>ror—ryrwrjr)r€rÐrrXrXrYÚget_featured_dataset¢s  (r‚z/reva/subject-idsc Csªz^t dtj›¡}|d}g}|D]&}|dddkr$| |dd¡q$td|dœƒd fWSty¤}z.t d |›¡td |d œƒd fWYd}~Sd}~00dS) Nú /packages/ÚchildrenræÚ packageTypeÚ Collectionr¼Úsuccess)r·ÚidsrÞz+Error while getting REVA subject id files: ©r·r¾éô) r~r“r>Ú,REVA_3D_TRACING_PRIMARY_FOLDER_COLLECTION_IDrIrryrwrj)Úprimary_folderÚprimary_childrenZ subject_idsrrlrXrXrYÚgetRevaSubjectIds³srŽz /reva/tracing-files/c sød‰d‰d‰zœt dtj›¡}|d}t‡fdd„|Dƒdƒ}|durtt dˆ›¡td d ˆ›d œƒd fWSt d|d d›¡}|d}t‡fdd„|Dƒdƒ}|durèt dˆ›dˆ›¡td ˆ›dˆ›d œƒd fWSt d|d d›¡}|d}t‡fdd„|Dƒdƒ} | dur^t dˆ›dˆ›¡td ˆ›dˆ›d œƒd fWSt d| d d›¡} | d} t‡fdd„| Dƒdƒ} | durÔt dˆ›dˆ›¡td ˆ›dˆ›d œƒd fWSt d| d d›¡} | d}g}|D]š}t d|d d›¡}|d}|D]n}|d d}t d|›d¡}|dd d}t d|›d|›¡d}| t |d dƒt |ƒdœ¡q$qütd|dœƒdfWSt yò}z.t d|›¡td|d œƒd fWYd}~Sd}~00dS)!NzCoordinates-DataZInSituz Vagus-nerverƒr„c3s"|]}|ddˆkr|VqdS©rær¼NrX©Ú.0r©Ú subject_idrXrYÚ Êr±z&getRevaTracingFiles..z/REVA tracing folder not found with subject id: ÚERRORz"Folder not found with subject id: r‰rirær]c3s"|]}|ddˆkr|VqdSrrXr)Úcoordinates_folder_namerXrYr”Ðr±zREVA tracing folder z not found for subject: ú folder not found for subject: c3s"|]}|ddˆkr|VqdSrrXr)Úin_situ_folder_namerXrYr”Ör±c3s"|]}|ddˆkr|VqdSrrXr)Úvagus_nerve_folder_namerXrYr”Ür±ú/viewrú/files/rr¼)r¼Ús3Urlr‡©r·rrÞz'Error while getting REVA tracing files z#Error while getting tracing files: rŠ) r~r“r>r‹ÚnextrwrjrrIrkry)r“rŒrÚ subject_childÚsubject_folderÚsubject_childrenZcoordinates_childZcoordinates_folderZcoordinates_childrenZ in_situ_childZin_situ_folderZin_situ_childrenZvagus_nerve_childZvagus_nerve_folderZvagus_nerve_childrenZvagus_tracing_filesZvagus_region_childZvagus_region_folderZvagus_region_childrenZvagus_file_childÚfile_package_idZ vagus_fileZ vagus_file_idZvagus_file_presigned_urlrlrX)r–r˜r“r™rYÚgetRevaTracingFilesÁsX   (r£z!/reva/micro-ct-files/c sˆ›d‰z¾t dtj›¡}|d}t‡fdd„|Dƒdƒ}|durrt dˆ›¡tddˆ›d œƒd fWSt d|d d ›¡}|d}t‡fd d„|Dƒdƒ}|duræt dˆ›dˆ›¡tdˆ›dˆ›d œƒd fWSt d|d d ›¡}|d}g} |D]¨} | d d } t d| ›d¡} | dd d } t d| ›d| ›¡d}| d d}| d}| d d}| dd d}|  t |ƒt |ƒt |ƒt |ƒt |ƒdœ¡qtd| dœƒdfWSt y}z.t d|›¡td|d œƒdfWYd}~Sd}~00dS)Nz-MicroCTVisualizationrƒr„c3s"|]}|ddˆkr|VqdSrrXrr’rXrYr”ùr±z&getRevaMicroCtFiles..z/REVA microCT folder not found with subject id: r•z*MicroCT folder not found with subject id: r‰rirær]c3s"|]}|ddˆkr|VqdSrrXr)Ú"micro_ct_visualization_folder_namerXrYr”ÿr±z REVA microCT r—ršrr›rr¼Zstorager…ZfileType)r¼rœrvr…r+r‡rrÞz'Error while getting REVA microCT files z#Error while getting microCT files: rŠ) r~r“r>Z*REVA_MICRO_CT_PRIMARY_FOLDER_COLLECTION_IDržrwrjrrIrkry)r“rŒrrŸr r¡Zmicro_ct_childZmicro_ct_visualization_folderZmicro_ct_childrenZmicro_ct_filesZ micro_childr¢Zmicro_child_fileZmicro_child_file_idZmicro_file_presigned_urlÚ file_nameÚ file_sizeZ package_typerhrlrX)r¤r“rYÚgetRevaMicroCtFilesòs>    .r§z/get_owner_email/csPtjj}tjj |¡}‡fdd„|Dƒ}|s:tdddntd|djiƒSdS)Ncsg|]}|jˆkr|‘qSrX)Zint_id)r‘Úx©Úowner_idrXrYÚ r±z#get_owner_email..rizOwner not foundrÙr½r)rsZ_apiZ _organizationZ organizationsZ get_membersrrr½)rªZorgÚmembersÚresrXr©rYÚget_owner_emails r®z!/get_body_scaffold_info/cCs:t|ƒ}|r(t|ƒ}tt|ƒƒ}|r(|Stdd|›dS)NrizWhole body info not found for rÙ)r4r-rCr0r)rEr]r2rJrXrXrYÚget_body_scaffold_info$s r¯z/thumbnail/Fc Csütƒ}z´t| ¡stƒWdƒn1s00Ytjd |¡}d| ¡i}tjd||d}t   |j ¡}|dkr¶|s¶t|  d¡Wdƒn1s¢0Yt |dƒ}|WStyð}zt d|›|¡WYd}~n d}~00td d ƒS) Nz/thumbnail/{0}r]rj©r–s¼eyJzdGF0dXMiOiJBZG1pbiB1c2VyIGF1dGhlbnRpY2F0aW9uIHJlcXVpcmVkIHRvIHZpZXcvZWRpdCB1c2VyIGluZm8uIFlvdSBtYXkgbmVlZCB0byBsb2cgb3V0IGFuZCBsb2cgYmFjayBpbiB0byByZXZlcmlmeSB5b3VyIGNyZWRlbnRpYWxzLiJ9rSTz Could not get the thumbnail for riz-An error occured while fetching the thumbnail)rRÚbiolucida_lockr]Úauthenticate_biolucidar>ÚBIOLUCIDA_ENDPOINTr`rtrrärårærZÚthumbnail_by_image_idryrwrjr)Úimage_idZrecursive_callÚblrr–rÐZencoded_contentrrXrXrYr´/s($ÿ ÿ( (r´z/image/c Csltjd |¡}zt d|¡}t| ¡ƒWSty`}zt  d|›|¡WYd}~n d}~00t ddƒS)Nz/image/info/{0}rjzCould not get image info for riz/An error occured while getting the image's info) r>r³r`rtrÚprocess_biolucida_resultr—ryrwrjr)rµrrÐrrXrXrYÚimage_info_by_image_idOs (r¸z/image_search/c Csjtjd |¡}zt d|¡}| ¡WSty\}zt d|›|¡WYd}~n d}~00ddidfS)Nz%/imagemap/search_dataset/discover/{0}rjz$Could not search images for dataset rjz3An error occured while searching images for datasetri) r>r³r`rtrr—ryrwrj)Z dataset_idrrÐrrXrXrYÚimage_search_by_dataset_idZs  (r¹z/image_xmp_info/cCsvtjd |¡}zt d|¡}Wn"tjjyBtdddYS0| ¡}|ddkrdt |dƒStdd |›dS) Nz/image/xmpmetadata/{0}rjr×ú)Unable to make a connection to Biolucida.rÙr·r‡rzXMP info not found for ) r>r³r`rtrrurçrr—Úprocess_biolucida_results©rµrrJrÐrXrXrYÚimage_xmp_infoes  r½z/image_blv_link/cCsztjd |¡}zt d|¡}Wn"tjjyBtdddYS0| ¡}|ddkrht d|diƒStdd |›dS) Nz/image/blv_link/{0}rjr×rºrÙr·r‡ÚlinkzBLV link not found for ) r>r³r`rtrrurçrr—rr¼rXrXrYÚimage_blv_linkts r¿cCsdtƒ}tjd}tjtjddœ}g}i}tjd||||d}|jtjj kr`|  ¡}|  |d¡dS)Nz /authenticaterS)ÚusernameÚpasswordr]rº)r–rrr]) rRr>r³ZBIOLUCIDA_USERNAMEZBIOLUCIDA_PASSWORDrtrrãZcodesr•r—rZ)r¶rrMrr–rÐrærXrXrYr²ƒs þr²cCshd}tjdrd}|rXt ¡}|rJd|vrJ|d}| ||¡}td|iƒStdddn td d ddS) NTÚTESTINGFÚstater^r×zState not specifiedrÙriúDatabase not available)ÚapprPrÚget_jsonZ pushStaterr)ržZcommitrprÃr^rXrXrYÚget_share_link”s    rÇcCs^|rNt ¡}|r@d|vr@|d}| |¡}|r@td| |¡iƒStdddn tddddS)Nr^rÃr×z#Key missing or did not find a matchrÙrirÄ)rrÆZ pullStaterr)ržrpr^rÃrXrXrYÚget_saved_state¤s  rÈz/map/getshareidcCsttƒSrT)rÇÚmaptablerXrXrXrYÚget_map_share_link²srÊz /map/getstatecCsttƒSrT)rÈrÉrXrXrXrYÚ get_map_state¸srËz/scaffold/getshareidcCsttƒSrT)rÇÚ scaffoldtablerXrXrXrYÚget_scaffold_share_link¾srÍz/scaffold/getstatecCsttƒSrT)rÈrÌrXrXrXrYÚget_scaffold_stateÄsrÎú/tasksc# CsJtj}d|vrˆzFtjtjtj|ddœd}| ¡}d|vsD|dsRddidfWSWqžty„}zt   d|¡WYd}~qžd}~00nt j d sžddidfS|r:d |vr:d |vr:|d }|d }|d }d d tj i}dtjd}tjtjtjtjtjtjg} tjtjtjtjtjtjg} tj} d} d} g}|rNd|vrN|d} | dkr’dtjd}tjtjtjg} tjtjtjg} tj} tj} | dkrØdtjd}tjtjtjg} tjtjtjg} tj} tj} nº| dkrdtjd}tjtjtjg} tjtjtjg} tj} tj} nt| dkrddtjd}tjtjtjg} tjtjtjg} tj} tj } n.| dkr’|  !tj"tj#g¡|  !tj"tj#g¡| dkrþd| }tj$||d}d| ¡vrþ| ¡dgkrþ| ¡ddd |}| ¡ddd}||| | | dddidœ}tj|||d}tj%}d| ¡vr®| ¡dgkr®| ¡ddd }|rðd!|vrð|d!}| &¡}|j'}|j(}d tj ||d"d#œ}d|d$}ztj|||d%Wn.tyî}zt)|ƒWYd}~n d}~00|D]¸}d|}tj$||d}d| ¡vrô| ¡dgkrô| ¡ddd | ¡ddd | ¡ddd&| ¡ddd'| ¡ddd(d|gddid)œ}tj|||dqô|j*d*kr0d+|vr|d+rd,|vr|d,d-krd.} t+ ,d/|i¡}!| d0krd1} t- ,d/|i¡}!nÐ| d2kr4d3} t. ,d/|i¡}!n²| d4krRd5} t/ ,d/|i¡}!n”| dkrpd6} t0 ,d/|i¡}!nv| dkrŽd7} t0 ,d/|i¡}!nX| dkr¬d8} t0 ,d/|i¡}!n:| dkrÊd9} t0 ,d/|i¡}!n| dkræd:} t0 ,d/|i¡}!|d+}"t1|"ƒdkrt2 3tj4|d+| |!¡t5||| ¡ddd d;S| ¡Sn t6ddS)?NZ captcha_token)ZsecretrÐ)rr—r‡rjzFailed Captcha Validationi™z0Could not validate captcha, bypassing validationrÂr:rÚÚ AuthorizationúBearer z%https://www.wrike.com/api/v4/folders/rÏrSrvÚnewsÚeventZtoolsAndResourcesZcommunitySpotlightZresearchz#https://www.wrike.com/api/v4/tasks/©rr–rrZ subTaskIdsFZBacklog)r:rÚÚ customStatusÚ followersÚ responsiblesÚfollowÚdates)rr—r–r]Ú attachmentZXMLHttpRequest)rÐz X-File-NamerzX-Requested-Withz /attachments)rrr–ZcustomStatusIdZ followerIdsZresponsibleIds)r:rÚrÕrÖr×rØZ superTasksrÙrÞÚ userEmailZsendCopyr z%SPARC Reported Error/Issue Submissionr¾ZfeedbackzSPARC Feedback SubmissionZinterestz!SPARC Service Interest SubmissionÚgeneralz$SPARC Question or Inquiry SubmissionzSPARC Research SubmissionzSPARC News SubmissionzSPARC Event SubmissionzSPARC Tool/Resource SubmissionzSPARC Story Submission)r:rÚÚtask_idr×zMissing title or descriptionrÙ)7rÚformrtr"r>Z TURNSTILE_URLZNUXT_TURNSTILE_SECRET_KEYr—ryrwrjrÅrPÚ WRIKE_TOKENZDRC_FEEDBACK_FOLDER_IDZCCB_HEAD_WRIKE_IDZDAT_CORE_TECH_LEAD_WRIKE_IDZMAP_CORE_TECH_LEAD_WRIKE_IDZK_CORE_TECH_LEAD_WRIKE_IDZSIM_CORE_TECH_LEAD_WRIKE_IDZMODERATOR_WRIKE_IDZDRC_WRIKE_CUSTOM_STATUS_IDZNEWS_AND_EVENTS_FOLDER_IDZCOMMS_LEAD_1_WRIKE_IDZCOMMS_LEAD_2_WRIKE_IDZCOMMS_LEAD_3_WRIKE_IDZCOMMS_WRIKE_CUSTOM_STATUS_IDZNEWS_TEMPLATE_TASK_IDZEVENT_TEMPLATE_TASK_IDZTOOLS_AND_RESOURCES_FOLDER_IDZ$TOOLS_AND_RESOURCES_TEMPLATE_TASK_IDZCOMMUNITY_SPOTLIGHT_FOLDER_IDZ$COMMUNITY_SPOTLIGHT_TEMPLATE_TASK_IDÚextendZ SUE_WRIKE_IDZ JYL_WRIKE_IDr“rrôÚfilenamerÎÚprintrãr9rÅr7r;r8r:rörÂrÃrÄrr)#rÞZ captchaReqZ captchaResprr:rÚZnewTaskDescriptionÚhedrrÖr×rÕZtaskTypeZtemplateTaskIdZtemplateSubTaskIdsZ templateUrlZ templateResprr[rZ new_task_idrÚZ file_datar¥rÎr–Zattachment_urlrlZ subTaskIdZsubTaskTemplateUrlZsubTaskTemplateRespZ subTaskDatarDÚbodyrÛrXrXrYÚcreate_wrike_taskÉs@þþ$        þ ù ý ü ý þ ø ý  ,        ý råz#/hubspot_contact_properties/c Cs8d|›d}ddtjdœ}zŒtj||d}|jdkrDt| ¡ƒWS|jdkrjtd d |›d d œƒdfWStd d|j›d|j d¡dkr’| ¡n|jdœƒ|jfWSWnŠtj yî}z,tdd|›dt |ƒdœƒdfWYd}~Sd}~0t y2}z,tdd|›dt |ƒdœƒdfWYd}~Sd}~00dS)Nz/https://api.hubapi.com/crm/v3/objects/contacts/zZ?archived=false&idProperty=email&properties=firstname,lastname,email,newsletter,event_namerŽrÑ©rßrÐr°rÞrizContact not foundzNo contact with the email 'z' was found in HubSpot.rSzFailed to fetch contactz'HubSpot API responded with status code Ú.rß)rjr¾Údetailsrèz"Could not get contact with email 'z' due to a request error.rŠzInternal Server ErrorzDAn unexpected error occurred while fetching the contact with email 'z'.) r>ÚHUBSPOT_API_TOKENrtr“rãrr—r–rúrèrkry)r½rr–rÐrrXrXrYÚget_hubspot_contact_properties€sL þ   þý ýü  ýü ýürêz/subscribe_to_newsletterc Cs˜tj}| d¡}| d¡}| d¡}|s8tddiƒdfSd}zHt|ƒ\}}|dkrd|d  d d¡}nt d |›¡td |›ƒ‚Wn:ty¾}z"t d |›d|›¡WYd}~n d}~00t|t ƒrâ|  d¡}t t d|ƒƒ}d|vrô|  d¡d|||d |¡dœ|ddœgi} d} ddtjdœ} z*tj| | | d} |  ¡t|  ¡ƒdfWStjjy’}z tdt |ƒiƒdfWYd}~Sd}~00dS)NÚ email_addressÚ first_nameÚ last_namerjzEmail is requiredr×rSrÞÚ propertiesÚ newsletterz"Unexpected response from HubSpot: zUnexpected error: z4Error while retrieving contact properties for email ú: ú;Z NewsletterZinputs)r½Ú firstnameÚlastnamerïr½)rîr]Z idPropertyz;https://api.hubapi.com/crm/v3/objects/contacts/batch/upsertrŽrÑræ©r—r–rŠ)rr—r“rrêrwrjryÚ isinstancerkÚsplitÚlistÚfilterrIÚjoinr>rértr"rârurè) rr½rìríZnewsletter_propertyZcontact_propertiesrãrlZcurrent_newsletter_valuesrMrr–rÐrXrXrYÚsubscribe_to_newsletter¨sR    ,   üøÿÿþrúc CsŒtjjtjd}z"|jjjjt |ƒgd¢dd}Wn6t yh}zt dd|›dWYd}~Sd}~00|szt dddS|j sŒt dd dS|j   d ¡s¤t dd dS|j   d ¡d j}|j   d ig¡d }|rÔ|jnd}|j   dig¡d }|rø|jnd}|j   d¡} t| ƒd kr| d } | r0| j d¡ng} |j   d¡} t| ƒd krV| d } | rh| j d¡ng} dd„| | Dƒ} |||| dœS)N)Z access_token)ròrór½rïÚ event_nameF)Z contact_idÚproperties_with_historyZarchivedr×z2Exception thrown when getting contact properties: rÙz-Failed to retrieve contact data from HubSpot.zContact properties not foundr½z Contact Email property not foundrròrSrórïrñrûcSsg|] }|r|‘qSrXrX©r‘ÚtagrXrXrYr«ÿr±z*get_contact_properties..)r½ròróÚtags)ÚhubspotZClientÚcreater>réZcrmZcontactsZ basic_apiZ get_by_idrkrrrür“rWrörö)Ú object_idÚclientÚ contact_datarlr½Zfirstname_dataròZ lastname_dataróZnewsletter_tags_dataZnewsletter_tagsZevents_tags_dataZ events_tagsrÿrXrXrYÚget_contact_propertiesâs<"(      ürc CsØd|›d}ddtjdœ}|||dœ||dœ}zHtj|||d} t| jƒd krpt d |›d | j›d | j›¡|   ¡WSt yÒ} zBt d |›d|›| ¡t dd|›d|›d| ›dWYd} ~ Sd} ~ 00dS)Nz#https://api.emailoctopus.com/lists/z /contactsrŽrÑræ)Z FirstNameZLastName)rër.r·rÿrôZ200z:Emailoctopus contact did not get added/updated for email: z. Returned a response of rðz4Could not add or update contact with email address: z in emailoctopus list: rŠz1Could not add/update contact with email address: z! from emailoctopus list with ID: ú due to the following error: rÙ) r>ZEMAIL_OCTOPUS_API_KEYrtÚputrkrãrwrjrúr—ryr) Zlist_idr½ròrórÿr·rr–rMrÐrrXrXrYÚ"add_or_update_emailoctopus_contacts" þü  rz/hubspot_webhookc sæt ¡}t|tƒrhzt |¡}WnFtjyf}z,t d|›¡t ddiƒdfWYd}~Sd}~00|s†t d¡t ddiƒdfSt j   dt›¡t j   d|›¡d |d vsÂd |d vrât d |›¡t dd iƒdfSdtj vsúdtj vr$t dtj ›¡t ddtj ›iƒdfStj  d¡}tj d}z t|ƒ}Wn4tyzt d|›¡t ddiƒdfYS0zÐtt ¡ƒ}||dkrÂt dtt›d|›¡t dhƒdfWStj}d}tj|dd}|›|›|›|›} t tj d¡|  d¡tj¡ ¡} t | ¡ d¡} t  | |¡sJt d¡t ddiƒdfWSWnLt!y˜} z2t d| ›¡t dd| ›iƒdfWYd} ~ Sd} ~ 00|d }|d ‰|d ‰d} ‡‡fdd „}t"j#|d! $¡t d"d#d$œƒd%fS)&Nz.Webhook request body has invalid JSON format: rjzInvalid JSON formatr×z'Webhook request does not contain a bodyzInvalid payloadz"Received Hubspot webhook request: zHubspot webhook request body: ZsubscriptionTyperZobjectIdz5Required keys missing in the following body payload: z Required keys missing in payloadzX-HubSpot-Request-TimestampzX-HubSpot-Signature-V3zKRequired signature header(s) not present in the following request headers: z$Invalid signature timestamp format: z"Invalid signature timestamp formatrOzqr±z.execute_webhook..cSsi|] }|d“qS)FrXrýrXrXrYr rr±zUnsupported subscription type: r‡zWebhook processed successfullyr‰rÞ) rÅZ app_contextrryrwrjrr>ZEMAIL_OCTOPUS_MASTER_LIST_IDrIÚupdater) rrròrór½Zemailoctopus_contactZ tags_to_addrþZtags_to_removeZupdated_contact_tags©rZsubscription_typerXrYÚexecute_webhook\s2  $       z(hubspot_webhook..execute_webhook)Útargetr‡z/Webhook request received and signature verifiedr‰éÌ)%rrÆrõrkr—rÀrUrwrjrrÅÚloggerr’r–r“rþrcÚtimeÚcurrentrr¸ÚhmacÚnewr>ÚHUBSPOT_CLIENT_SECRETÚencodeÚhashlibÚsha256ÚdigesträrårõZcompare_digestryÚ threadingÚThreadrœ)rärlZsignature_headerZtimestamp_headerZsignature_timestampZ current_timerrnZstringified_bodyZ raw_stringZhashed_signatureZbase64_hashed_signaturerrrrXrrYÚhubspot_webhooksj &      ý ,rz/mailchimp_subscribec Csæt ¡}|rÖd|vrÖd|vrÖd|vrÖ|d}|d}|d}tdtjƒ}d|}|d||dœdœ}z6tj|||d }|jd krˆ| ¡WSd | ¡WSWn0t yÈ}zt   d |¡WYd}~n d}~00d didfSt ddddS)NrërìríÚAnyUserú;https://us2.api.mailchimp.com/3.0/lists/c81a347bd8/members/r )ZFNAMEZLNAME)rër·Z merge_fields©rr—ÚauthrÞz(Failed to subscribe user with response: z!Could not subscribe to newsletterrjzÚMAILCHIMP_API_KEYrtrrãr—ryrwrjr) rprërìrír#rrr[rrXrXrYÚsubscribe_to_mailchimp{s4 þýý  " r&z/mailchimp_unsubscribec Cs¼t ¡}|r¬d|vr¬|d}tdtjƒ}d|}ddi}z6tj|||d}|jdkr^| ¡WSd| ¡WSWn0t yž}zt   d |¡WYd}~n d}~00d d id fSt d dddS)Nrër r!r·Z unsubscribedr"rÞz*Failed to unsubscribe user with response: z#Could not unsubscribe to newsletterrjz>An error occured while trying to unsubscribe to the newsletterrŠr×úMissing email_addressrÙr$)rprër#rrr[rrXrXrYÚunsubscribe_to_mailchimpŸs(  ÿý  " r(z&/mailchimp_member_info/c Cs |rtdtjƒ}d|}z4tj||d}|jdkr<| ¡WSd| ¡WSWn6ty‚}zt  d|›|¡WYd}~n d}~00ddid fSt d d d dS) Nr r!)rr#rÞz)Failed to get member info with response: zFailed to get member info for rjz(Could not get member info from MailChimprŠr×r'rÙ) r%r>r%rtr“rãr—ryrwrjr)rër#rr[rrXrXrYÚget_mailchimp_member_info»s þ  ( r)z/get-organ-curies/c CsŒtj d¡}t|ƒ}i}z,tjtj›dtj›|d}t |  ¡ƒ}Wn>t y‚}z&t   d|¡dddœdfWYd}~Sd}~00t|ƒS) NrEr r!zFailed getting Uberon IDsrTÚ BaseExceptionrVrá)rrÒr-r/rtr"r>r#r$rEr—r*rwrjr)rEZ requestBodyrJrÐrrXrXrYÚget_available_uberonidsÓs" þ þýr+z/get-related-terms/c Cs°tjjdddtjjdddtjjdddtjdœ}i}z*tjtj›d |›|d }t| ¡ƒ}WnDt y¦}z,t   d |›|¡d d dœdfWYd}~Sd}~00t |ƒS)NÚ directionZOUTGOING)ruÚrelationshipTypez BFO:0000050Úentailr )r,r-r.r}z/graph/neighbors/)rÜz*Failed getting related terms with payload rTr*rVrá) rrÒr“r>r$rtZSCI_CRUNCH_SCIGRAPH_HOSTrGr—r*rwrjr)r2rMrJrÐrrXrXrYÚget_related_termsës(üþþýr/z /simulation_ui_file/cCs¤ttt|ƒƒƒ}t |j¡}zd|dd}|d}|dddd}t dd|›d |›¡}t d d|¡}tt t ||ƒ¡ƒWSt yžt d d d Yn0dS)NrHrZs3urizabi-simulation-fileZdatasetrÖz s3://[^/]*/rSzfiles/z s3://|/.*riz$no simulation UI file could be foundrÙ) rBr0r-r—rÀrÚreÚsubrrryr)r=rHZ results_jsonÚitemrrÖrÍZs3_bucket_namerXrXrYÚsimulation_ui_files   r3z /pmr_filecCs€t ¡}|rpd|vrpz@t tj›d|d›¡}|jdkrHt |j ¡WS|  ¡WSWq|t dddYq|0n t ddddS)NrÖrrÞr×z invalid pathrÙz missing path) rrÆrtr"r>ZPMR_HOSTrãrärårær—r)rr[rXrXrYÚpmr_files  r4z/start_simulationcCsJt ¡}|r:d|vr:d|dvr:d|dvr:t t|ƒ¡StddddS)NÚsolverr¼rÕr×z)Missing solver name and/or solver versionrÙ)rrÆr—r¸Údo_start_simulationr©rrXrXrYrL's$rLz/check_simulationcCsRt ¡}|rBd|vrBd|vrBd|dvrBd|dvrBt t|ƒ¡StddddS)NZjob_idr5r¼rÕr×z1Missing solver name, solver version and/or job idrÙ)rrÆr—r¸Údo_check_simulationrr7rXrXrYrM1s,rMz/pmr_latest_exposurecCs°t ¡}|r d|vr zptj|dddid}|jdkrxz$| ¡dddd dd }Wnd }Yn0t|d WS| ¡WSWq¬td ddYq¬0n td dddS)NZ workspace_urlÚAcceptz$application/vnd.physiome.pmr2.json.1r°rÞÚ collectionÚitemsrZlinksZhrefrSrŒr×zInvalid workspace URLrÙzMissing workspace URL)rrÆrtr“rãr—rr)rr[rrXrXrYÚpmr_latest_exposure;s$  ÿ $ ÿr<z/onto_term_lookupc CsÂtj d¡}ddi}dtji}t|ƒ}zbtjtj›d|||d}| ¡}|dd}|dd}|d kr||d }|d } nd d i} | WSt y¸} zt   d| ¡WYd} ~ n d} ~ 00t dƒS)NrRr9rŽr}rL)r–rÜr—r{Útotalr©rÚ_sourceÚlabelz not foundz.An error occured while fetching from SciCrunchrŠ) rrÒr“r>r$r0rtZSCI_CRUNCH_INTERLEX_HOSTr—ryrwrjr) rRr–rÜr2rÐrHr{r=rJrprrXrXrYÚfind_by_onto_termUs& ÿÿ   "r@z/search-readme/c Csxd|}ddtji}ztj||d}| ¡WStjjyr}z(t |¡t |ƒddœdfWYd}~Sd}~00dS)Nz2https://dash.readme.com/api/v1/docs/search?search=rÐzBasic rÔz9Readme is not currently reachable, please try again laterrSrá) r>ZREADME_API_KEYrtr"r—rurvrwrjrk)r2rr–rÐr{rXrXrYÚ search_readmeusþ  þýrAz/metricscCstSrT)r¡rXrXrXrYÚmetrics‡srBz/event_updatedcCsrtj d¡}|tjkr*tdd|›dnDt ¡}|rbz t|ƒWStdd|›dYqn0n tddddS)NZevent_updated_secret_keyrzInvalid secret key: rÙr×zInvalid event data: zMissing event data)rr–r“r>ZCTF_CDA_ACCESS_TOKENrrÆr)Z secret_keyrÓrXrXrYÚ event_updatedŒs   rCz/all_dataset_idscCs"tƒ}dd„|Dƒ}d}| |¡S)NcSsg|] }t|ƒ‘qSrX)rk)r‘ÚelementrXrXrYr«Ÿr±z#all_dataset_ids..z, )r rù)r÷Z string_listÚ delimiterrXrXrYÚall_dataset_idsœsrF)rNr)F( ÚatexiträZapp.metrics.pennsieverZapp.metrics.contentfulrrrZ!scripts.update_contentful_entriesrrZapp.metrics.algoliarr r r Zapp.metrics.gar r Zscripts.monthly_statsrZ"scripts.update_featured_dataset_idrrZapp.osparc.servicesrrZboto3rrrrZhubspot.crm.contactsrr—rwr0rtrÚ urllib.parserZ!apscheduler.schedulers.backgroundrZapscheduler.triggers.combiningrZapscheduler.triggers.daterZapscheduler.triggers.intervalrZbotocore.exceptionsrrrZflaskrrrrZ flask_corsrZflask_marshmallowr Z pennsiever!Z pennsieve2Z Pennsieve2Zpennsieve2.directr"Zpennsieve.baser#rxZPILr$Ú requests.authr%Z flask_cachingr&Zapp.scicrunch_requestsr'r(r)r*r+r,r-r.r/r0r1r2r3r4r5Zscripts.email_senderr6r7r8r9r:r;r<Z xml.etreer=Ú app.configr>Z app.dbtabler?r@rAZapp.scicrunch_process_resultsrBrCrDrErFrGrHZapp.serializerrIZ app.utilitiesrJrKZapp.osparc.osparcrLr6rMr8Zapp.biolucida_process_resultsr»rNr·Ú basicConfigrdrÅZ LOG_LEVELÚupperÚ log_levelrÚsetLevelÚgetattrÚWARNINGZ DEPLOY_ENVrPÚcacheZmarÂrsrZSPARC_PORTAL_AWS_KEYZSPARC_PORTAL_AWS_SECRETrQr±Z DATABASE_URLrÉÚAttributeErrorrÌr¯ÚobjectrRZ errorhandlerrmZbefore_first_requestr|rršr¤rµr´r³r›r’rœZmonthly_stats_email_schedulerÚmsZmonthly_stats_required_checkZadd_jobrar™rŸr¡r r¢r£r¥r¦r§Zfeatured_dataset_id_triggerr¶ÚregisterZrouter¹rÆrÑZDEFAULT_S3_BUCKET_NAMErÔrêrr rrrrr&r(r/r3r5r7r9r;r?rArBrFrKr0rQrXrZr\rgr˜rqrrrsrxrzr}rÚcachedr‚rŽr£r§r®r¯r´r¸r¹r½r¿r²rÇrÈrÊrËrÍrÎrårêrúrrrr&r(r)r+r/r3r4r<r@rArBrCrFrXrXrXrYÚs*                   D    $     ü          ÿ         >  '           ,   0            0 "         7'9 & ]#