![[Jenkins]](/jenkins/static/36965a9f/images/svgs/logo.svg){kind=logo}
Started by timer Running as SYSTEM Building remotely on Ubuntu_18.04_bioeng49 (buildslave Testing) in workspace /home/cmiss/Jenkins/workspace/12-Labours-API [WS-CLEANUP] Deleting project workspace... [WS-CLEANUP] Deferred wipeout is used... [WS-CLEANUP] Done The recommended git tool is: NONE No credentials specified Cloning the remote Git repository Cloning repository https://github.com/ABI-Software/12-labours-api.git > git init /home/cmiss/Jenkins/workspace/12-Labours-API # timeout=10 Fetching upstream changes from https://github.com/ABI-Software/12-labours-api.git > git --version # timeout=10 > git --version # 'git version 2.25.1' > git fetch --tags --force --progress -- https://github.com/ABI-Software/12-labours-api.git +refs/heads/*:refs/remotes/origin/* # timeout=10 > git config remote.origin.url https://github.com/ABI-Software/12-labours-api.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10 Avoid second fetch > git rev-parse refs/remotes/origin/dev^{commit} # timeout=10 Checking out Revision fed7495860d0492a0d83f51077693f3de98e748d (refs/remotes/origin/dev) > git config core.sparsecheckout # timeout=10 > git checkout -f fed7495860d0492a0d83f51077693f3de98e748d # timeout=10 Commit message: "Merge pull request #43 from ddjnw1yu/access-control" > git rev-list --no-walk fed7495860d0492a0d83f51077693f3de98e748d # timeout=10 [12-Labours-API] $ /bin/sh -xe /tmp/shiningpanda14988263072010979299.sh + pwd + export PYTHONPATH=/home/cmiss/Jenkins/workspace/12-Labours-API + pip install -r requirements.txt Requirement already satisfied: fastapi==0.80.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements.txt (line 1)) (0.80.0) Requirement already satisfied: fastapi-utils==0.2.1 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements.txt (line 2)) (0.2.1) Requirement already satisfied: gen3==4.19.1 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements.txt (line 3)) (4.19.1) Requirement already satisfied: gunicorn==20.1.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements.txt (line 4)) (20.1.0) Requirement already satisfied: PyJWT==2.7.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements.txt (line 5)) (2.7.0) Requirement already satisfied: pyorthanc==1.11.5 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements.txt (line 6)) (1.11.5) Requirement already satisfied: python-dotenv==0.20.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements.txt (line 7)) (0.20.0) Requirement already satisfied: python-irodsclient==1.1.8 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements.txt (line 8)) (1.1.8) Requirement already satisfied: sgqlc==16.3 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements.txt (line 9)) (16.3) Requirement already satisfied: uvicorn==0.18.2 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements.txt (line 10)) (0.18.2) Requirement already satisfied: starlette==0.19.1 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from fastapi==0.80.0->-r requirements.txt (line 1)) (0.19.1) Requirement already satisfied: pydantic!=1.7,!=1.7.1,!=1.7.2,!=1.7.3,!=1.8,!=1.8.1,<2.0.0,>=1.6.2 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from fastapi==0.80.0->-r requirements.txt (line 1)) (1.10.15) Requirement already satisfied: sqlalchemy<2.0.0,>=1.3.12 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from fastapi-utils==0.2.1->-r requirements.txt (line 2)) (1.4.52) Requirement already satisfied: aiofiles<0.9.0,>=0.8.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.19.1->-r requirements.txt (line 3)) (0.8.0) Requirement already satisfied: aiohttp in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.19.1->-r requirements.txt (line 3)) (3.9.5) Requirement already satisfied: backoff in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.19.1->-r requirements.txt (line 3)) (1.11.1) Requirement already satisfied: cdislogging<2.0.0,>=1.1.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.19.1->-r requirements.txt (line 3)) (1.1.1) Requirement already satisfied: click in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.19.1->-r requirements.txt (line 3)) (8.1.7) Requirement already satisfied: dataclasses-json in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.19.1->-r requirements.txt (line 3)) (0.6.4) Requirement already satisfied: drsclient<0.3.0,>=0.2.2 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.19.1->-r requirements.txt (line 3)) (0.2.3) Requirement already satisfied: httpx in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.19.1->-r requirements.txt (line 3)) (0.23.3) Requirement already satisfied: humanfriendly in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.19.1->-r requirements.txt (line 3)) (10.0) Requirement already satisfied: indexclient in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.19.1->-r requirements.txt (line 3)) (2.2.1) Requirement already satisfied: jsonschema in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.19.1->-r requirements.txt (line 3)) (3.2.0) Requirement already satisfied: pandas<2.0.0,>=1.4.2 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.19.1->-r requirements.txt (line 3)) (1.5.3) Requirement already satisfied: pypfb<1.0.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.19.1->-r requirements.txt (line 3)) (0.5.29) Requirement already satisfied: python-dateutil in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.19.1->-r requirements.txt (line 3)) (2.9.0.post0) Requirement already satisfied: requests in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.19.1->-r requirements.txt (line 3)) (2.31.0) Requirement already satisfied: tqdm<5.0.0,>=4.61.2 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.19.1->-r requirements.txt (line 3)) (4.66.2) Requirement already satisfied: urllib3<2.0.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.19.1->-r requirements.txt (line 3)) (1.26.18) Requirement already satisfied: xmltodict<0.14.0,>=0.13.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.19.1->-r requirements.txt (line 3)) (0.13.0) Requirement already satisfied: setuptools>=3.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gunicorn==20.1.0->-r requirements.txt (line 4)) (69.5.1) Requirement already satisfied: pydicom<3.0.0,>=2.3.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pyorthanc==1.11.5->-r requirements.txt (line 6)) (2.4.4) Requirement already satisfied: PrettyTable>=0.7.2 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from python-irodsclient==1.1.8->-r requirements.txt (line 8)) (3.10.0) Requirement already satisfied: defusedxml in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from python-irodsclient==1.1.8->-r requirements.txt (line 8)) (0.7.1) Requirement already satisfied: six>=1.10.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from python-irodsclient==1.1.8->-r requirements.txt (line 8)) (1.16.0) Requirement already satisfied: graphql-core<4.0.0,>=3.1.7 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from sgqlc==16.3->-r requirements.txt (line 9)) (3.2.3) Requirement already satisfied: h11>=0.8 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from uvicorn==0.18.2->-r requirements.txt (line 10)) (0.14.0) Requirement already satisfied: anyio<5,>=3.4.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from starlette==0.19.1->fastapi==0.80.0->-r requirements.txt (line 1)) (3.6.2) Requirement already satisfied: typing-extensions>=3.10.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from starlette==0.19.1->fastapi==0.80.0->-r requirements.txt (line 1)) (4.11.0) Requirement already satisfied: asyncio<4.0.0,>=3.4.3 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from drsclient<0.3.0,>=0.2.2->gen3==4.19.1->-r requirements.txt (line 3)) (3.4.3) Requirement already satisfied: certifi in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from httpx->gen3==4.19.1->-r requirements.txt (line 3)) (2024.2.2) Requirement already satisfied: httpcore<0.17.0,>=0.15.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from httpx->gen3==4.19.1->-r requirements.txt (line 3)) (0.16.3) Requirement already satisfied: rfc3986<2,>=1.3 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from rfc3986[idna2008]<2,>=1.3->httpx->gen3==4.19.1->-r requirements.txt (line 3)) (1.5.0) Requirement already satisfied: sniffio in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from httpx->gen3==4.19.1->-r requirements.txt (line 3)) (1.3.1) Requirement already satisfied: pytz>=2020.1 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pandas<2.0.0,>=1.4.2->gen3==4.19.1->-r requirements.txt (line 3)) (2024.1) Requirement already satisfied: numpy>=1.20.3 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pandas<2.0.0,>=1.4.2->gen3==4.19.1->-r requirements.txt (line 3)) (1.26.4) Requirement already satisfied: wcwidth in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from PrettyTable>=0.7.2->python-irodsclient==1.1.8->-r requirements.txt (line 8)) (0.2.13) Requirement already satisfied: PyYAML<7.0.0,>=6.0.1 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pypfb<1.0.0->gen3==4.19.1->-r requirements.txt (line 3)) (6.0.1) Requirement already satisfied: dictionaryutils<4.0.0,>=3.4.8 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pypfb<1.0.0->gen3==4.19.1->-r requirements.txt (line 3)) (3.4.10) Requirement already satisfied: fastavro<1.9.0,>=1.8.2 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pypfb<1.0.0->gen3==4.19.1->-r requirements.txt (line 3)) (1.8.4) Requirement already satisfied: gen3dictionary>=2.0.3 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pypfb<1.0.0->gen3==4.19.1->-r requirements.txt (line 3)) (2.0.3) Requirement already satisfied: python-json-logger<0.2.0,>=0.1.11 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pypfb<1.0.0->gen3==4.19.1->-r requirements.txt (line 3)) (0.1.11) Requirement already satisfied: aiosignal>=1.1.2 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from aiohttp->gen3==4.19.1->-r requirements.txt (line 3)) (1.3.1) Requirement already satisfied: attrs>=17.3.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from aiohttp->gen3==4.19.1->-r requirements.txt (line 3)) (23.2.0) Requirement already satisfied: frozenlist>=1.1.1 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from aiohttp->gen3==4.19.1->-r requirements.txt (line 3)) (1.4.1) Requirement already satisfied: multidict<7.0,>=4.5 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from aiohttp->gen3==4.19.1->-r requirements.txt (line 3)) (6.0.5) Requirement already satisfied: yarl<2.0,>=1.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from aiohttp->gen3==4.19.1->-r requirements.txt (line 3)) (1.9.4) Requirement already satisfied: async-timeout<5.0,>=4.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from aiohttp->gen3==4.19.1->-r requirements.txt (line 3)) (4.0.3) Requirement already satisfied: charset-normalizer<4,>=2 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from requests->gen3==4.19.1->-r requirements.txt (line 3)) (3.3.2) Requirement already satisfied: idna<4,>=2.5 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from requests->gen3==4.19.1->-r requirements.txt (line 3)) (3.7) Requirement already satisfied: greenlet!=0.4.17 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from sqlalchemy<2.0.0,>=1.3.12->fastapi-utils==0.2.1->-r requirements.txt (line 2)) (3.0.3) Requirement already satisfied: marshmallow<4.0.0,>=3.18.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from dataclasses-json->gen3==4.19.1->-r requirements.txt (line 3)) (3.21.1) Requirement already satisfied: typing-inspect<1,>=0.4.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from dataclasses-json->gen3==4.19.1->-r requirements.txt (line 3)) (0.9.0) Requirement already satisfied: pyrsistent>=0.14.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from jsonschema->gen3==4.19.1->-r requirements.txt (line 3)) (0.20.0) Requirement already satisfied: packaging>=17.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from marshmallow<4.0.0,>=3.18.0->dataclasses-json->gen3==4.19.1->-r requirements.txt (line 3)) (24.0) Requirement already satisfied: mypy-extensions>=0.3.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from typing-inspect<1,>=0.4.0->dataclasses-json->gen3==4.19.1->-r requirements.txt (line 3)) (1.0.0) [notice] A new release of pip is available: 24.0 -> 24.2 [notice] To update, run: pip install --upgrade pip + pip install -r requirements-dev.txt Requirement already satisfied: pytest==7.1.2 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements-dev.txt (line 1)) (7.1.2) Requirement already satisfied: pytest-timeout==2.1.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements-dev.txt (line 2)) (2.1.0) Requirement already satisfied: anyio==3.6.2 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements-dev.txt (line 3)) (3.6.2) Requirement already satisfied: attrs>=19.2.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pytest==7.1.2->-r requirements-dev.txt (line 1)) (23.2.0) Requirement already satisfied: iniconfig in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pytest==7.1.2->-r requirements-dev.txt (line 1)) (2.0.0) Requirement already satisfied: packaging in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pytest==7.1.2->-r requirements-dev.txt (line 1)) (24.0) Requirement already satisfied: pluggy<2.0,>=0.12 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pytest==7.1.2->-r requirements-dev.txt (line 1)) (1.5.0) Requirement already satisfied: py>=1.8.2 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pytest==7.1.2->-r requirements-dev.txt (line 1)) (1.11.0) Requirement already satisfied: tomli>=1.0.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pytest==7.1.2->-r requirements-dev.txt (line 1)) (2.0.1) Requirement already satisfied: idna>=2.8 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from anyio==3.6.2->-r requirements-dev.txt (line 3)) (3.7) Requirement already satisfied: sniffio>=1.1 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from anyio==3.6.2->-r requirements-dev.txt (line 3)) (1.3.1) [notice] A new release of pip is available: 24.0 -> 24.2 [notice] To update, run: pip install --upgrade pip + pytest ============================= test session starts ============================== platform linux -- Python 3.9.19, pytest-7.1.2, pluggy-1.5.0 rootdir: /home/cmiss/Jenkins/workspace/12-Labours-API plugins: anyio-3.6.2, timeout-2.1.0 collected 39 items tests/test_api/test_access_api.py ... [ 7%] tests/test_api/test_gen3_api.py FFF. [ 17%] tests/test_api/test_irods_api.py FF [ 23%] tests/test_api/test_orthanc_api.py .. [ 28%] tests/test_data/test_gen3_data.py FFFF [ 38%] tests/test_data/test_irods_data.py F [ 41%] tests/test_function/test_filter/test_filter_editor.py .. [ 46%] tests/test_function/test_filter/test_filter_formatter.py .... [ 56%] tests/test_function/test_filter/test_filter_generator.py ... [ 64%] tests/test_function/test_filter/test_filter_logic.py ... [ 71%] tests/test_function/test_pagination/test_pagination_formatter.py . [ 74%] tests/test_function/test_query/test_query_formatter.py ...... [ 89%] tests/test_function/test_search/test_search_logic.py .... [100%] =================================== FAILURES =================================== _____________________________ test_get_gen3_record _____________________________ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff541f0a430> method = 'POST', url = '/user/credentials/cdis/access_token' body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...NPva0i6TdvsONgWYV1N3mET3cX78Px5n6SH84sTMCLqWr5EOzw4ImhcQ2kOYfMOqKA", "key_id": "92c93027-24d6-4240-825c-30b9395d78cf"}' headers = {'User-Agent': 'python-requests/2.31.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'} retries = Retry(total=0, connect=None, read=False, redirect=None, status=None) redirect = False, assert_same_host = False timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None release_conn = False, chunked = False, body_pos = None response_kw = {'decode_content': False, 'preload_content': False} parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None) destination_scheme = None, conn = None, release_this_conn = True http_tunnel_required = False, err = None, clean_exit = False def urlopen( self, method, url, body=None, headers=None, retries=None, redirect=True, assert_same_host=True, timeout=_Default, pool_timeout=None, release_conn=None, chunked=False, body_pos=None, **response_kw ): """ Get a connection from the pool and perform an HTTP request. This is the lowest level call for making a request, so you'll need to specify all the raw details. .. note:: More commonly, it's appropriate to use a convenience method provided by :class:`.RequestMethods`, such as :meth:`request`. .. note:: `release_conn` will only behave as expected if `preload_content=False` because we want to make `preload_content=False` the default behaviour someday soon without breaking backwards compatibility. :param method: HTTP request method (such as GET, POST, PUT, etc.) :param url: The URL to perform the request on. :param body: Data to send in the request body, either :class:`str`, :class:`bytes`, an iterable of :class:`str`/:class:`bytes`, or a file-like object. :param headers: Dictionary of custom headers to send, such as User-Agent, If-None-Match, etc. If None, pool headers are used. If provided, these headers completely replace any pool-specific headers. :param retries: Configure the number of retries to allow before raising a :class:`~urllib3.exceptions.MaxRetryError` exception. Pass ``None`` to retry until you receive a response. Pass a :class:`~urllib3.util.retry.Retry` object for fine-grained control over different types of retries. Pass an integer number to retry connection errors that many times, but no other types of errors. Pass zero to never retry. If ``False``, then retries are disabled and any exception is raised immediately. Also, instead of raising a MaxRetryError on redirects, the redirect response will be returned. :type retries: :class:`~urllib3.util.retry.Retry`, False, or an int. :param redirect: If True, automatically handle redirects (status codes 301, 302, 303, 307, 308). Each redirect counts as a retry. Disabling retries will disable redirect, too. :param assert_same_host: If ``True``, will make sure that the host of the pool requests is consistent else will raise HostChangedError. When ``False``, you can use the pool on an HTTP proxy and request foreign hosts. :param timeout: If specified, overrides the default timeout for this one request. It may be a float (in seconds) or an instance of :class:`urllib3.util.Timeout`. :param pool_timeout: If set and the pool is set to block=True, then this method will block for ``pool_timeout`` seconds and raise EmptyPoolError if no connection is available within the time period. :param release_conn: If False, then the urlopen call will not release the connection back into the pool once a response is received (but will release if you read the entire contents of the response such as when `preload_content=True`). This is useful if you're not preloading the response's content immediately. You will need to call ``r.release_conn()`` on the response ``r`` to return the connection back into the pool. If None, it takes the value of ``response_kw.get('preload_content', True)``. :param chunked: If True, urllib3 will send the body using chunked transfer encoding. Otherwise, urllib3 will send the body using the standard content-length form. Defaults to False. :param int body_pos: Position to seek to in file-like body in the event of a retry or redirect. Typically this won't need to be set because urllib3 will auto-populate the value when needed. :param \\**response_kw: Additional parameters are passed to :meth:`urllib3.response.HTTPResponse.from_httplib` """ parsed_url = parse_url(url) destination_scheme = parsed_url.scheme if headers is None: headers = self.headers if not isinstance(retries, Retry): retries = Retry.from_int(retries, redirect=redirect, default=self.retries) if release_conn is None: release_conn = response_kw.get("preload_content", True) # Check host if assert_same_host and not self.is_same_host(url): raise HostChangedError(self, url, retries) # Ensure that the URL we're connecting to is properly encoded if url.startswith("/"): url = six.ensure_str(_encode_target(url)) else: url = six.ensure_str(parsed_url.url) conn = None # Track whether `conn` needs to be released before # returning/raising/recursing. Update this variable if necessary, and # leave `release_conn` constant throughout the function. That way, if # the function recurses, the original value of `release_conn` will be # passed down into the recursive call, and its value will be respected. # # See issue #651 [1] for details. # # [1] <https://github.com/urllib3/urllib3/issues/651> release_this_conn = release_conn http_tunnel_required = connection_requires_http_tunnel( self.proxy, self.proxy_config, destination_scheme ) # Merge the proxy headers. Only done when not using HTTP CONNECT. We # have to copy the headers dict so we can safely change it without those # changes being reflected in anyone else's copy. if not http_tunnel_required: headers = headers.copy() headers.update(self.proxy_headers) # Must keep the exception bound to a separate variable or else Python 3 # complains about UnboundLocalError. err = None # Keep track of whether we cleanly exited the except block. This # ensures we do proper cleanup in finally. clean_exit = False # Rewind body position, if needed. Record current position # for future rewinds in the event of a redirect/retry. body_pos = set_file_position(body, body_pos) try: # Request a connection from the queue. timeout_obj = self._get_timeout(timeout) conn = self._get_conn(timeout=pool_timeout) conn.timeout = timeout_obj.connect_timeout is_new_proxy_conn = self.proxy is not None and not getattr( conn, "sock", None ) if is_new_proxy_conn and http_tunnel_required: self._prepare_proxy(conn) # Make the request on the httplib connection object. > httplib_response = self._make_request( conn, method, url, timeout=timeout_obj, body=body, headers=headers, chunked=chunked, ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:715: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff541f0a430> conn = <urllib3.connection.HTTPSConnection object at 0x7ff541f0ab20> method = 'POST', url = '/user/credentials/cdis/access_token' timeout = Timeout(connect=None, read=None, total=None), chunked = False httplib_request_kw = {'body': b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwd...p, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}} timeout_obj = Timeout(connect=None, read=None, total=None) def _make_request( self, conn, method, url, timeout=_Default, chunked=False, **httplib_request_kw ): """ Perform a request on a given urllib connection object taken from our pool. :param conn: a connection from one of our connection pools :param timeout: Socket timeout in seconds for the request. This can be a float or integer, which will set the same timeout value for the socket connect and the socket read, or an instance of :class:`urllib3.util.Timeout`, which gives you more fine-grained control over your timeouts. """ self.num_requests += 1 timeout_obj = self._get_timeout(timeout) timeout_obj.start_connect() conn.timeout = Timeout.resolve_default_timeout(timeout_obj.connect_timeout) # Trigger any extra validation we need to do. try: > self._validate_conn(conn) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:404: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff541f0a430> conn = <urllib3.connection.HTTPSConnection object at 0x7ff541f0ab20> def _validate_conn(self, conn): """ Called right before a request is made, after the socket is created. """ super(HTTPSConnectionPool, self)._validate_conn(conn) # Force connect early to allow us to validate the connection. if not getattr(conn, "sock", None): # AppEngine might not have `.sock` > conn.connect() ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:1058: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <urllib3.connection.HTTPSConnection object at 0x7ff541f0ab20> def connect(self): # Add certificate verification self.sock = conn = self._new_conn() hostname = self.host tls_in_tls = False if self._is_using_tunnel(): if self.tls_in_tls_required: self.sock = conn = self._connect_tls_proxy(hostname, conn) tls_in_tls = True # Calls self._set_hostport(), so self.host is # self._tunnel_host below. self._tunnel() # Mark this connection as not reusable self.auto_open = 0 # Override the host with the one we're requesting data from. hostname = self._tunnel_host server_hostname = hostname if self.server_hostname is not None: server_hostname = self.server_hostname is_time_off = datetime.date.today() < RECENT_DATE if is_time_off: warnings.warn( ( "System time is way off (before {0}). This will probably " "lead to SSL verification errors" ).format(RECENT_DATE), SystemTimeWarning, ) # Wrap socket using verification with the root certs in # trusted_root_certs default_ssl_context = False if self.ssl_context is None: default_ssl_context = True self.ssl_context = create_urllib3_context( ssl_version=resolve_ssl_version(self.ssl_version), cert_reqs=resolve_cert_reqs(self.cert_reqs), ) context = self.ssl_context context.verify_mode = resolve_cert_reqs(self.cert_reqs) # Try to load OS default certs if none are given. # Works well on Windows (requires Python3.4+) if ( not self.ca_certs and not self.ca_cert_dir and not self.ca_cert_data and default_ssl_context and hasattr(context, "load_default_certs") ): context.load_default_certs() > self.sock = ssl_wrap_socket( sock=conn, keyfile=self.key_file, certfile=self.cert_file, key_password=self.key_password, ca_certs=self.ca_certs, ca_cert_dir=self.ca_cert_dir, ca_cert_data=self.ca_cert_data, server_hostname=server_hostname, ssl_context=context, tls_in_tls=tls_in_tls, ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:419: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> keyfile = None, certfile = None, cert_reqs = None ca_certs = '/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/certifi/cacert.pem' server_hostname = 'gen3.abi-ctt-ctp.cloud.edu.au', ssl_version = None ciphers = None, ssl_context = <ssl.SSLContext object at 0x7ff541f23740> ca_cert_dir = None, key_password = None, ca_cert_data = None, tls_in_tls = False def ssl_wrap_socket( sock, keyfile=None, certfile=None, cert_reqs=None, ca_certs=None, server_hostname=None, ssl_version=None, ciphers=None, ssl_context=None, ca_cert_dir=None, key_password=None, ca_cert_data=None, tls_in_tls=False, ): """ All arguments except for server_hostname, ssl_context, and ca_cert_dir have the same meaning as they do when using :func:`ssl.wrap_socket`. :param server_hostname: When SNI is supported, the expected hostname of the certificate :param ssl_context: A pre-made :class:`SSLContext` object. If none is provided, one will be created using :func:`create_urllib3_context`. :param ciphers: A string of ciphers we wish the client to support. :param ca_cert_dir: A directory containing CA certificates in multiple separate files, as supported by OpenSSL's -CApath flag or the capath argument to SSLContext.load_verify_locations(). :param key_password: Optional password if the keyfile is encrypted. :param ca_cert_data: Optional string containing CA certificates in PEM format suitable for passing as the cadata parameter to SSLContext.load_verify_locations() :param tls_in_tls: Use SSLTransport to wrap the existing socket. """ context = ssl_context if context is None: # Note: This branch of code and all the variables in it are no longer # used by urllib3 itself. We should consider deprecating and removing # this code. context = create_urllib3_context(ssl_version, cert_reqs, ciphers=ciphers) if ca_certs or ca_cert_dir or ca_cert_data: try: context.load_verify_locations(ca_certs, ca_cert_dir, ca_cert_data) except (IOError, OSError) as e: raise SSLError(e) elif ssl_context is None and hasattr(context, "load_default_certs"): # try to load OS default certs; works well on Windows (require Python3.4+) context.load_default_certs() # Attempt to detect if we get the goofy behavior of the # keyfile being encrypted and OpenSSL asking for the # passphrase via the terminal and instead error out. if keyfile and key_password is None and _is_key_file_encrypted(keyfile): raise SSLError("Client private key is encrypted, password is required") if certfile: if key_password is None: context.load_cert_chain(certfile, keyfile) else: context.load_cert_chain(certfile, keyfile, key_password) try: if hasattr(context, "set_alpn_protocols"): context.set_alpn_protocols(ALPN_PROTOCOLS) except NotImplementedError: # Defensive: in CI, we always have set_alpn_protocols pass # If we detect server_hostname is an IP address then the SNI # extension should not be used according to RFC3546 Section 3.1 use_sni_hostname = server_hostname and not is_ipaddress(server_hostname) # SecureTransport uses server_hostname in certificate verification. send_sni = (use_sni_hostname and HAS_SNI) or ( IS_SECURETRANSPORT and server_hostname ) # Do not warn the user if server_hostname is an invalid SNI hostname. if not HAS_SNI and use_sni_hostname: warnings.warn( "An HTTPS request has been made, but the SNI (Server Name " "Indication) extension to TLS is not available on this platform. " "This may cause the server to present an incorrect TLS " "certificate, which can cause validation failures. You can upgrade to " "a newer version of Python to solve this. For more information, see " "https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html" "#ssl-warnings", SNIMissingWarning, ) if send_sni: > ssl_sock = _ssl_wrap_socket_impl( sock, context, tls_in_tls, server_hostname=server_hostname ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/ssl_.py:449: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> ssl_context = <ssl.SSLContext object at 0x7ff541f23740>, tls_in_tls = False server_hostname = 'gen3.abi-ctt-ctp.cloud.edu.au' def _ssl_wrap_socket_impl(sock, ssl_context, tls_in_tls, server_hostname=None): if tls_in_tls: if not SSLTransport: # Import error, ssl is not available. raise ProxySchemeUnsupported( "TLS in TLS requires support for the 'ssl' module" ) SSLTransport._validate_ssl_context_for_tls_in_tls(ssl_context) return SSLTransport(sock, ssl_context, server_hostname) if server_hostname: > return ssl_context.wrap_socket(sock, server_hostname=server_hostname) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/ssl_.py:493: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <ssl.SSLContext object at 0x7ff541f23740> sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> server_side = False, do_handshake_on_connect = True, suppress_ragged_eofs = True server_hostname = 'gen3.abi-ctt-ctp.cloud.edu.au', session = None def wrap_socket(self, sock, server_side=False, do_handshake_on_connect=True, suppress_ragged_eofs=True, server_hostname=None, session=None): # SSLSocket class handles server_hostname encoding before it calls # ctx._wrap_socket() > return self.sslsocket_class._create( sock=sock, server_side=server_side, do_handshake_on_connect=do_handshake_on_connect, suppress_ragged_eofs=suppress_ragged_eofs, server_hostname=server_hostname, context=self, session=session ) /usr/lib/python3.9/ssl.py:501: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ cls = <class 'ssl.SSLSocket'> sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> server_side = False, do_handshake_on_connect = True, suppress_ragged_eofs = True server_hostname = 'gen3.abi-ctt-ctp.cloud.edu.au' context = <ssl.SSLContext object at 0x7ff541f23740>, session = None @classmethod def _create(cls, sock, server_side=False, do_handshake_on_connect=True, suppress_ragged_eofs=True, server_hostname=None, context=None, session=None): if sock.getsockopt(SOL_SOCKET, SO_TYPE) != SOCK_STREAM: raise NotImplementedError("only stream sockets are supported") if server_side: if server_hostname: raise ValueError("server_hostname can only be specified " "in client mode") if session is not None: raise ValueError("session can only be specified in " "client mode") if context.check_hostname and not server_hostname: raise ValueError("check_hostname requires server_hostname") kwargs = dict( family=sock.family, type=sock.type, proto=sock.proto, fileno=sock.fileno() ) self = cls.__new__(cls, **kwargs) super(SSLSocket, self).__init__(**kwargs) sock_timeout = sock.gettimeout() sock.detach() self._context = context self._session = session self._closed = False self._sslobj = None self.server_side = server_side self.server_hostname = context._encode_hostname(server_hostname) self.do_handshake_on_connect = do_handshake_on_connect self.suppress_ragged_eofs = suppress_ragged_eofs # See if we are connected try: self.getpeername() except OSError as e: if e.errno != errno.ENOTCONN: raise connected = False blocking = self.getblocking() self.setblocking(False) try: # We are not connected so this is not supposed to block, but # testing revealed otherwise on macOS and Windows so we do # the non-blocking dance regardless. Our raise when any data # is found means consuming the data is harmless. notconn_pre_handshake_data = self.recv(1) except OSError as e: # EINVAL occurs for recv(1) on non-connected on unix sockets. if e.errno not in (errno.ENOTCONN, errno.EINVAL): raise notconn_pre_handshake_data = b'' self.setblocking(blocking) if notconn_pre_handshake_data: # This prevents pending data sent to the socket before it was # closed from escaping to the caller who could otherwise # presume it came through a successful TLS connection. reason = "Closed before TLS handshake with data in recv buffer." notconn_pre_handshake_data_error = SSLError(e.errno, reason) # Add the SSLError attributes that _ssl.c always adds. notconn_pre_handshake_data_error.reason = reason notconn_pre_handshake_data_error.library = None try: self.close() except OSError: pass try: raise notconn_pre_handshake_data_error finally: # Explicitly break the reference cycle. notconn_pre_handshake_data_error = None else: connected = True self.settimeout(sock_timeout) # Must come after setblocking() calls. self._connected = connected if connected: # create the SSL object try: self._sslobj = self._context._wrap_socket( self, server_side, self.server_hostname, owner=self, session=self._session, ) if do_handshake_on_connect: timeout = self.gettimeout() if timeout == 0.0: # non-blocking raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets") > self.do_handshake() /usr/lib/python3.9/ssl.py:1074: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <ssl.SSLSocket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> block = False @_sslcopydoc def do_handshake(self, block=False): self._check_connected() timeout = self.gettimeout() try: if timeout == 0.0 and block: self.settimeout(None) > self._sslobj.do_handshake() E ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133) /usr/lib/python3.9/ssl.py:1343: SSLCertVerificationError During handling of the above exception, another exception occurred: self = <requests.adapters.HTTPAdapter object at 0x7ff541f0a280> request = <PreparedRequest [POST]>, stream = False timeout = Timeout(connect=None, read=None, total=None), verify = True cert = None, proxies = OrderedDict() def send( self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None ): """Sends PreparedRequest object. Returns Response object. :param request: The :class:`PreparedRequest <PreparedRequest>` being sent. :param stream: (optional) Whether to stream the request content. :param timeout: (optional) How long to wait for the server to send data before giving up, as a float, or a :ref:`(connect timeout, read timeout) <timeouts>` tuple. :type timeout: float or tuple or urllib3 Timeout object :param verify: (optional) Either a boolean, in which case it controls whether we verify the server's TLS certificate, or a string, in which case it must be a path to a CA bundle to use :param cert: (optional) Any user-provided SSL certificate to be trusted. :param proxies: (optional) The proxies dictionary to apply to the request. :rtype: requests.Response """ try: conn = self.get_connection(request.url, proxies) except LocationValueError as e: raise InvalidURL(e, request=request) self.cert_verify(conn, request.url, verify, cert) url = self.request_url(request, proxies) self.add_headers( request, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies, ) chunked = not (request.body is None or "Content-Length" in request.headers) if isinstance(timeout, tuple): try: connect, read = timeout timeout = TimeoutSauce(connect=connect, read=read) except ValueError: raise ValueError( f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, " f"or a single float to set both timeouts to the same value." ) elif isinstance(timeout, TimeoutSauce): pass else: timeout = TimeoutSauce(connect=timeout, read=timeout) try: > resp = conn.urlopen( method=request.method, url=url, body=request.body, headers=request.headers, redirect=False, assert_same_host=False, preload_content=False, decode_content=False, retries=self.max_retries, timeout=timeout, chunked=chunked, ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:486: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff541f0a430> method = 'POST', url = '/user/credentials/cdis/access_token' body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...NPva0i6TdvsONgWYV1N3mET3cX78Px5n6SH84sTMCLqWr5EOzw4ImhcQ2kOYfMOqKA", "key_id": "92c93027-24d6-4240-825c-30b9395d78cf"}' headers = {'User-Agent': 'python-requests/2.31.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'} retries = Retry(total=0, connect=None, read=False, redirect=None, status=None) redirect = False, assert_same_host = False timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None release_conn = False, chunked = False, body_pos = None response_kw = {'decode_content': False, 'preload_content': False} parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None) destination_scheme = None, conn = None, release_this_conn = True http_tunnel_required = False, err = None, clean_exit = False def urlopen( self, method, url, body=None, headers=None, retries=None, redirect=True, assert_same_host=True, timeout=_Default, pool_timeout=None, release_conn=None, chunked=False, body_pos=None, **response_kw ): """ Get a connection from the pool and perform an HTTP request. This is the lowest level call for making a request, so you'll need to specify all the raw details. .. note:: More commonly, it's appropriate to use a convenience method provided by :class:`.RequestMethods`, such as :meth:`request`. .. note:: `release_conn` will only behave as expected if `preload_content=False` because we want to make `preload_content=False` the default behaviour someday soon without breaking backwards compatibility. :param method: HTTP request method (such as GET, POST, PUT, etc.) :param url: The URL to perform the request on. :param body: Data to send in the request body, either :class:`str`, :class:`bytes`, an iterable of :class:`str`/:class:`bytes`, or a file-like object. :param headers: Dictionary of custom headers to send, such as User-Agent, If-None-Match, etc. If None, pool headers are used. If provided, these headers completely replace any pool-specific headers. :param retries: Configure the number of retries to allow before raising a :class:`~urllib3.exceptions.MaxRetryError` exception. Pass ``None`` to retry until you receive a response. Pass a :class:`~urllib3.util.retry.Retry` object for fine-grained control over different types of retries. Pass an integer number to retry connection errors that many times, but no other types of errors. Pass zero to never retry. If ``False``, then retries are disabled and any exception is raised immediately. Also, instead of raising a MaxRetryError on redirects, the redirect response will be returned. :type retries: :class:`~urllib3.util.retry.Retry`, False, or an int. :param redirect: If True, automatically handle redirects (status codes 301, 302, 303, 307, 308). Each redirect counts as a retry. Disabling retries will disable redirect, too. :param assert_same_host: If ``True``, will make sure that the host of the pool requests is consistent else will raise HostChangedError. When ``False``, you can use the pool on an HTTP proxy and request foreign hosts. :param timeout: If specified, overrides the default timeout for this one request. It may be a float (in seconds) or an instance of :class:`urllib3.util.Timeout`. :param pool_timeout: If set and the pool is set to block=True, then this method will block for ``pool_timeout`` seconds and raise EmptyPoolError if no connection is available within the time period. :param release_conn: If False, then the urlopen call will not release the connection back into the pool once a response is received (but will release if you read the entire contents of the response such as when `preload_content=True`). This is useful if you're not preloading the response's content immediately. You will need to call ``r.release_conn()`` on the response ``r`` to return the connection back into the pool. If None, it takes the value of ``response_kw.get('preload_content', True)``. :param chunked: If True, urllib3 will send the body using chunked transfer encoding. Otherwise, urllib3 will send the body using the standard content-length form. Defaults to False. :param int body_pos: Position to seek to in file-like body in the event of a retry or redirect. Typically this won't need to be set because urllib3 will auto-populate the value when needed. :param \\**response_kw: Additional parameters are passed to :meth:`urllib3.response.HTTPResponse.from_httplib` """ parsed_url = parse_url(url) destination_scheme = parsed_url.scheme if headers is None: headers = self.headers if not isinstance(retries, Retry): retries = Retry.from_int(retries, redirect=redirect, default=self.retries) if release_conn is None: release_conn = response_kw.get("preload_content", True) # Check host if assert_same_host and not self.is_same_host(url): raise HostChangedError(self, url, retries) # Ensure that the URL we're connecting to is properly encoded if url.startswith("/"): url = six.ensure_str(_encode_target(url)) else: url = six.ensure_str(parsed_url.url) conn = None # Track whether `conn` needs to be released before # returning/raising/recursing. Update this variable if necessary, and # leave `release_conn` constant throughout the function. That way, if # the function recurses, the original value of `release_conn` will be # passed down into the recursive call, and its value will be respected. # # See issue #651 [1] for details. # # [1] <https://github.com/urllib3/urllib3/issues/651> release_this_conn = release_conn http_tunnel_required = connection_requires_http_tunnel( self.proxy, self.proxy_config, destination_scheme ) # Merge the proxy headers. Only done when not using HTTP CONNECT. We # have to copy the headers dict so we can safely change it without those # changes being reflected in anyone else's copy. if not http_tunnel_required: headers = headers.copy() headers.update(self.proxy_headers) # Must keep the exception bound to a separate variable or else Python 3 # complains about UnboundLocalError. err = None # Keep track of whether we cleanly exited the except block. This # ensures we do proper cleanup in finally. clean_exit = False # Rewind body position, if needed. Record current position # for future rewinds in the event of a redirect/retry. body_pos = set_file_position(body, body_pos) try: # Request a connection from the queue. timeout_obj = self._get_timeout(timeout) conn = self._get_conn(timeout=pool_timeout) conn.timeout = timeout_obj.connect_timeout is_new_proxy_conn = self.proxy is not None and not getattr( conn, "sock", None ) if is_new_proxy_conn and http_tunnel_required: self._prepare_proxy(conn) # Make the request on the httplib connection object. httplib_response = self._make_request( conn, method, url, timeout=timeout_obj, body=body, headers=headers, chunked=chunked, ) # If we're going to release the connection in ``finally:``, then # the response doesn't need to know about the connection. Otherwise # it will also try to release it and we'll have a double-release # mess. response_conn = conn if not release_conn else None # Pass method to Response for length checking response_kw["request_method"] = method # Import httplib's response into our own wrapper object response = self.ResponseCls.from_httplib( httplib_response, pool=self, connection=response_conn, retries=retries, **response_kw ) # Everything went great! clean_exit = True except EmptyPoolError: # Didn't get a connection from the pool, no need to clean up clean_exit = True release_this_conn = False raise except ( TimeoutError, HTTPException, SocketError, ProtocolError, BaseSSLError, SSLError, CertificateError, ) as e: # Discard the connection for these exceptions. It will be # replaced during the next _get_conn() call. clean_exit = False def _is_ssl_error_message_from_http_proxy(ssl_error): # We're trying to detect the message 'WRONG_VERSION_NUMBER' but # SSLErrors are kinda all over the place when it comes to the message, # so we try to cover our bases here! message = " ".join(re.split("[^a-z]", str(ssl_error).lower())) return ( "wrong version number" in message or "unknown protocol" in message ) # Try to detect a common user error with proxies which is to # set an HTTP proxy to be HTTPS when it should be 'http://' # (ie {'http': 'http://proxy', 'https': 'https://proxy'}) # Instead we add a nice error message and point to a URL. if ( isinstance(e, BaseSSLError) and self.proxy and _is_ssl_error_message_from_http_proxy(e) and conn.proxy and conn.proxy.scheme == "https" ): e = ProxyError( "Your proxy appears to only use HTTP and not HTTPS, " "try changing your proxy URL to be HTTP. See: " "https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html" "#https-proxy-error-http-proxy", SSLError(e), ) elif isinstance(e, (BaseSSLError, CertificateError)): e = SSLError(e) elif isinstance(e, (SocketError, NewConnectionError)) and self.proxy: e = ProxyError("Cannot connect to proxy.", e) elif isinstance(e, (SocketError, HTTPException)): e = ProtocolError("Connection aborted.", e) > retries = retries.increment( method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:799: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = Retry(total=0, connect=None, read=False, redirect=None, status=None) method = 'POST', url = '/user/credentials/cdis/access_token', response = None error = SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133)')) _pool = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff541f0a430> _stacktrace = <traceback object at 0x7ff54200e500> def increment( self, method=None, url=None, response=None, error=None, _pool=None, _stacktrace=None, ): """Return a new Retry object with incremented retry counters. :param response: A response object, or None, if the server did not return a response. :type response: :class:`~urllib3.response.HTTPResponse` :param Exception error: An error encountered during the request, or None if the response was received successfully. :return: A new ``Retry`` object. """ if self.total is False and error: # Disabled, indicate to re-raise the error. raise six.reraise(type(error), error, _stacktrace) total = self.total if total is not None: total -= 1 connect = self.connect read = self.read redirect = self.redirect status_count = self.status other = self.other cause = "unknown" status = None redirect_location = None if error and self._is_connection_error(error): # Connect retry? if connect is False: raise six.reraise(type(error), error, _stacktrace) elif connect is not None: connect -= 1 elif error and self._is_read_error(error): # Read retry? if read is False or not self._is_method_retryable(method): raise six.reraise(type(error), error, _stacktrace) elif read is not None: read -= 1 elif error: # Other retry? if other is not None: other -= 1 elif response and response.get_redirect_location(): # Redirect retry? if redirect is not None: redirect -= 1 cause = "too many redirects" redirect_location = response.get_redirect_location() status = response.status else: # Incrementing because of a server error like a 500 in # status_forcelist and the given method is in the allowed_methods cause = ResponseError.GENERIC_ERROR if response and response.status: if status_count is not None: status_count -= 1 cause = ResponseError.SPECIFIC_ERROR.format(status_code=response.status) status = response.status history = self.history + ( RequestHistory(method, url, error, status, redirect_location), ) new_retry = self.new( total=total, connect=connect, read=read, redirect=redirect, status=status_count, other=other, history=history, ) if new_retry.is_exhausted(): > raise MaxRetryError(_pool, url, error or ResponseError(cause)) E urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133)'))) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/retry.py:592: MaxRetryError During handling of the above exception, another exception occurred: client = <starlette.testclient.TestClient object at 0x7ff541f1df40> token = {'access_token': 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZGVudGl0eSI6InB1YmxpYyIsInNjb3BlIjpbImRlbW8xLTEyTCJdLCJleHBpcmUiOiJOb25lIn0.doEgWg8P2u6dGhajC1CBwJHS3TtbJh4L_vhMfK75uJQ'} def test_get_gen3_record(client, token): UUID = "5b9ae1bd-e780-4869-a458-b3422084c480" > response = client.get( f"/record/{UUID}", headers={"Authorization": f"Bearer {token['access_token']}"}, ) tests/test_api/test_gen3_api.py:26: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:602: in get return self.request("GET", url, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:476: in request return super().request( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:589: in request resp = self.send(prep, **send_kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:703: in send r = adapter.send(request, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:270: in send raise exc ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:267: in send portal.call(self.app, scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:283: in call return cast(T_Retval, self.start_task_soon(func, *args).result()) /usr/lib/python3.9/concurrent/futures/_base.py:446: in result return self.__get_result() /usr/lib/python3.9/concurrent/futures/_base.py:391: in __get_result raise self._exception ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:219: in _call_func retval = await retval ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/applications.py:269: in __call__ await super().__call__(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/applications.py:124: in __call__ await self.middleware_stack(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:184: in __call__ raise exc ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:162: in __call__ await self.app(scope, receive, _send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/cors.py:84: in __call__ await self.app(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:93: in __call__ raise exc ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:82: in __call__ await self.app(scope, receive, sender) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:21: in __call__ raise e ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:18: in __call__ await self.app(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:670: in __call__ await route.handle(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:266: in handle await self.app(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:65: in app response = await func(request) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:227: in app raw_response = await run_endpoint_function( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:160: in run_endpoint_function return await dependant.call(**values) app/main.py:318: in get_gen3_record record = connection["gen3"].export_record(program, project, uuid, "json") ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/submission.py:348: in export_record output = requests.get(api_url, auth=self._auth_provider).text ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:73: in get return request("get", url, params=params, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request return session.request(method=method, url=url, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:575: in request prep = self.prepare_request(req) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:486: in prepare_request p.prepare( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:372: in prepare self.prepare_auth(auth, url) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:603: in prepare_auth r = auth(self) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:325: in __call__ request.headers["Authorization"] = self._get_auth_value() ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:446: in _get_auth_value return "bearer " + self.get_access_token() ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:433: in get_access_token return self.refresh_access_token( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:367: in refresh_access_token self._access_token = get_access_token_with_key(self._refresh_token) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:75: in get_access_token_with_key resp = requests.post(auth_url, json=api_key) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:115: in post return request("post", url, data=data, json=json, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request return session.request(method=method, url=url, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:589: in request resp = self.send(prep, **send_kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:703: in send r = adapter.send(request, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <requests.adapters.HTTPAdapter object at 0x7ff541f0a280> request = <PreparedRequest [POST]>, stream = False timeout = Timeout(connect=None, read=None, total=None), verify = True cert = None, proxies = OrderedDict() def send( self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None ): """Sends PreparedRequest object. Returns Response object. :param request: The :class:`PreparedRequest <PreparedRequest>` being sent. :param stream: (optional) Whether to stream the request content. :param timeout: (optional) How long to wait for the server to send data before giving up, as a float, or a :ref:`(connect timeout, read timeout) <timeouts>` tuple. :type timeout: float or tuple or urllib3 Timeout object :param verify: (optional) Either a boolean, in which case it controls whether we verify the server's TLS certificate, or a string, in which case it must be a path to a CA bundle to use :param cert: (optional) Any user-provided SSL certificate to be trusted. :param proxies: (optional) The proxies dictionary to apply to the request. :rtype: requests.Response """ try: conn = self.get_connection(request.url, proxies) except LocationValueError as e: raise InvalidURL(e, request=request) self.cert_verify(conn, request.url, verify, cert) url = self.request_url(request, proxies) self.add_headers( request, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies, ) chunked = not (request.body is None or "Content-Length" in request.headers) if isinstance(timeout, tuple): try: connect, read = timeout timeout = TimeoutSauce(connect=connect, read=read) except ValueError: raise ValueError( f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, " f"or a single float to set both timeouts to the same value." ) elif isinstance(timeout, TimeoutSauce): pass else: timeout = TimeoutSauce(connect=timeout, read=timeout) try: resp = conn.urlopen( method=request.method, url=url, body=request.body, headers=request.headers, redirect=False, assert_same_host=False, preload_content=False, decode_content=False, retries=self.max_retries, timeout=timeout, chunked=chunked, ) except (ProtocolError, OSError) as err: raise ConnectionError(err, request=request) except MaxRetryError as e: if isinstance(e.reason, ConnectTimeoutError): # TODO: Remove this in 3.0.0: see #2811 if not isinstance(e.reason, NewConnectionError): raise ConnectTimeout(e, request=request) if isinstance(e.reason, ResponseError): raise RetryError(e, request=request) if isinstance(e.reason, _ProxyError): raise ProxyError(e, request=request) if isinstance(e.reason, _SSLError): # This branch is for urllib3 v1.22 and later. > raise SSLError(e, request=request) E requests.exceptions.SSLError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133)'))) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:517: SSLError ------------------------------ Captured log setup ------------------------------ ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. INFO app.main:main.py:166 {'gen3': False, 'irods': True, 'orthanc': True} WARNING app.main:main.py:185 Failed to update default filter. ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. ------------------------------ Captured log call ------------------------------- ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. _________________________ test_get_gen3_graphql_query __________________________ client = <starlette.testclient.TestClient object at 0x7ff5403b5ac0> token = {'access_token': 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZGVudGl0eSI6InB1YmxpYyIsInNjb3BlIjpbImRlbW8xLTEyTCJdLCJleHBpcmUiOiJOb25lIn0.doEgWg8P2u6dGhajC1CBwJHS3TtbJh4L_vhMfK75uJQ'} def test_get_gen3_graphql_query(client, token): DATASET_ID = "dataset-217-version-2" pass_case = { "node": "experiment_query", "filter": {"submitter_id": [DATASET_ID]}, "search": "", } > response = client.post( "/graphql/query?mode=data", json=pass_case, headers={"Authorization": f"Bearer {token['access_token']}"}, ) tests/test_api/test_gen3_api.py:54: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:637: in post return self.request("POST", url, data=data, json=json, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:476: in request return super().request( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:589: in request resp = self.send(prep, **send_kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:703: in send r = adapter.send(request, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:270: in send raise exc ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:267: in send portal.call(self.app, scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:283: in call return cast(T_Retval, self.start_task_soon(func, *args).result()) /usr/lib/python3.9/concurrent/futures/_base.py:446: in result return self.__get_result() /usr/lib/python3.9/concurrent/futures/_base.py:391: in __get_result raise self._exception ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:219: in _call_func retval = await retval ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/applications.py:269: in __call__ await super().__call__(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/applications.py:124: in __call__ await self.middleware_stack(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:184: in __call__ raise exc ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:162: in __call__ await self.app(scope, receive, _send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/cors.py:84: in __call__ await self.app(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:93: in __call__ raise exc ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:82: in __call__ await self.app(scope, receive, sender) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:21: in __call__ raise e ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:18: in __call__ await self.app(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:670: in __call__ await route.handle(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:266: in handle await self.app(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:65: in app response = await func(request) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:227: in app raw_response = await run_endpoint_function( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:160: in run_endpoint_function return await dependant.call(**values) app/main.py:416: in get_gen3_graphql_query query_result = QL.get_query_data(item) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <app.function.query.query_logic.QueryLogic object at 0x7ff542148610> item = GraphQLQueryItem(node='experiment_query', page=None, limit=None, filter={'submitter_id': ['dataset-217-version-2']}, search='', access=['demo1-12L'], asc=None, desc=None) def get_query_data(self, item): """ Handler for fetching data based on query item """ items = self._process_query_item(item) # Assume there will have maximum two datasets have same submitter id at current stage # One for public, another one for private # Show private dataset by default if user has the authority fetch_result = self._handle_thread_fetch(items) if "private" in fetch_result and fetch_result["private"] != []: return fetch_result["private"] > return fetch_result["public"] E KeyError: 'public' app/function/query/query_logic.py:69: KeyError ------------------------------ Captured log setup ------------------------------ ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. INFO app.main:main.py:166 {'gen3': False, 'irods': True, 'orthanc': True} WARNING app.main:main.py:185 Failed to update default filter. ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. ------------------------------ Captured log call ------------------------------- ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. _______________________ test_get_gen3_graphql_pagination _______________________ client = <starlette.testclient.TestClient object at 0x7ff540ae5dc0> token = {'access_token': 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZGVudGl0eSI6InB1YmxpYyIsInNjb3BlIjpbImRlbW8xLTEyTCJdLCJleHBpcmUiOiJOb25lIn0.doEgWg8P2u6dGhajC1CBwJHS3TtbJh4L_vhMfK75uJQ'} def test_get_gen3_graphql_pagination(client, token): filter_pass_case = { "filter": { "dataset_description_filter>study_organ_system": ["Stomach", "Vagus nerve"], "manifest_filter>additional_types": ["Plot"], "case_filter>species": ["Rat"], "case_filter>sex": ["Male"], } } response = client.post( "/graphql/pagination", json=filter_pass_case, headers={"Authorization": f"Bearer {token['access_token']}"}, ) result = response.json() > assert response.status_code == 200 E assert 400 == 200 E + where 400 = <Response [400]>.status_code tests/test_api/test_gen3_api.py:167: AssertionError ------------------------------ Captured log setup ------------------------------ ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. INFO app.main:main.py:166 {'gen3': False, 'irods': True, 'orthanc': True} WARNING app.main:main.py:185 Failed to update default filter. ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. ------------------------------ Captured log call ------------------------------- ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. __________________________ test_get_irods_collection ___________________________ client = <starlette.testclient.TestClient object at 0x7ff540e2ad60> token = {'access_token': 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZGVudGl0eSI6InB1YmxpYyIsInNjb3BlIjpbImRlbW8xLTEyTCJdLCJleHBpcmUiOiJOb25lIn0.doEgWg8P2u6dGhajC1CBwJHS3TtbJh4L_vhMfK75uJQ'} def test_get_irods_collection(client, token): pass_case_root = {} response = client.post( "/collection", json=pass_case_root, headers={"Authorization": f"Bearer {token['access_token']}"}, ) result = response.json() > assert response.status_code == 200 E assert 404 == 200 E + where 404 = <Response [404]>.status_code tests/test_api/test_irods_api.py:41: AssertionError ------------------------------ Captured log setup ------------------------------ ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. INFO app.main:main.py:166 {'gen3': False, 'irods': True, 'orthanc': True} WARNING app.main:main.py:185 Failed to update default filter. ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. ------------------------------ Captured log call ------------------------------- ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. ___________________________ test_get_irods_data_file ___________________________ client = <starlette.testclient.TestClient object at 0x7ff540bfe610> one_off_token = {'one_off_token': 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZGVudGl0eSI6InB1YmxpYyIsImV4cCI6MTcyNTgyMjg4Nn0.qR4mO5JYw5hP-8xQn5ewC17zcjlvx1XRxfA1PGF5C_k'} def test_get_irods_data_file(client, one_off_token): ACTION = "preview" FILEPATH = "dataset-217-version-2/derivative/scaffold_context_info.json" response = client.get( f"/data/{ACTION}/{FILEPATH}?token={one_off_token['one_off_token']}" ) result = response.json() > assert response.status_code == 200 E assert 404 == 200 E + where 404 = <Response [404]>.status_code tests/test_api/test_irods_api.py:92: AssertionError ------------------------------ Captured log setup ------------------------------ ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. INFO app.main:main.py:166 {'gen3': False, 'irods': True, 'orthanc': True} WARNING app.main:main.py:185 Failed to update default filter. ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. ------------------------------ Captured log call ------------------------------- ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. _____________________________ test_experiment_node _____________________________ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff54053b0a0> method = 'POST', url = '/user/credentials/cdis/access_token' body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...NPva0i6TdvsONgWYV1N3mET3cX78Px5n6SH84sTMCLqWr5EOzw4ImhcQ2kOYfMOqKA", "key_id": "92c93027-24d6-4240-825c-30b9395d78cf"}' headers = {'User-Agent': 'python-requests/2.31.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'} retries = Retry(total=0, connect=None, read=False, redirect=None, status=None) redirect = False, assert_same_host = False timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None release_conn = False, chunked = False, body_pos = None response_kw = {'decode_content': False, 'preload_content': False} parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None) destination_scheme = None, conn = None, release_this_conn = True http_tunnel_required = False, err = None, clean_exit = False def urlopen( self, method, url, body=None, headers=None, retries=None, redirect=True, assert_same_host=True, timeout=_Default, pool_timeout=None, release_conn=None, chunked=False, body_pos=None, **response_kw ): """ Get a connection from the pool and perform an HTTP request. This is the lowest level call for making a request, so you'll need to specify all the raw details. .. note:: More commonly, it's appropriate to use a convenience method provided by :class:`.RequestMethods`, such as :meth:`request`. .. note:: `release_conn` will only behave as expected if `preload_content=False` because we want to make `preload_content=False` the default behaviour someday soon without breaking backwards compatibility. :param method: HTTP request method (such as GET, POST, PUT, etc.) :param url: The URL to perform the request on. :param body: Data to send in the request body, either :class:`str`, :class:`bytes`, an iterable of :class:`str`/:class:`bytes`, or a file-like object. :param headers: Dictionary of custom headers to send, such as User-Agent, If-None-Match, etc. If None, pool headers are used. If provided, these headers completely replace any pool-specific headers. :param retries: Configure the number of retries to allow before raising a :class:`~urllib3.exceptions.MaxRetryError` exception. Pass ``None`` to retry until you receive a response. Pass a :class:`~urllib3.util.retry.Retry` object for fine-grained control over different types of retries. Pass an integer number to retry connection errors that many times, but no other types of errors. Pass zero to never retry. If ``False``, then retries are disabled and any exception is raised immediately. Also, instead of raising a MaxRetryError on redirects, the redirect response will be returned. :type retries: :class:`~urllib3.util.retry.Retry`, False, or an int. :param redirect: If True, automatically handle redirects (status codes 301, 302, 303, 307, 308). Each redirect counts as a retry. Disabling retries will disable redirect, too. :param assert_same_host: If ``True``, will make sure that the host of the pool requests is consistent else will raise HostChangedError. When ``False``, you can use the pool on an HTTP proxy and request foreign hosts. :param timeout: If specified, overrides the default timeout for this one request. It may be a float (in seconds) or an instance of :class:`urllib3.util.Timeout`. :param pool_timeout: If set and the pool is set to block=True, then this method will block for ``pool_timeout`` seconds and raise EmptyPoolError if no connection is available within the time period. :param release_conn: If False, then the urlopen call will not release the connection back into the pool once a response is received (but will release if you read the entire contents of the response such as when `preload_content=True`). This is useful if you're not preloading the response's content immediately. You will need to call ``r.release_conn()`` on the response ``r`` to return the connection back into the pool. If None, it takes the value of ``response_kw.get('preload_content', True)``. :param chunked: If True, urllib3 will send the body using chunked transfer encoding. Otherwise, urllib3 will send the body using the standard content-length form. Defaults to False. :param int body_pos: Position to seek to in file-like body in the event of a retry or redirect. Typically this won't need to be set because urllib3 will auto-populate the value when needed. :param \\**response_kw: Additional parameters are passed to :meth:`urllib3.response.HTTPResponse.from_httplib` """ parsed_url = parse_url(url) destination_scheme = parsed_url.scheme if headers is None: headers = self.headers if not isinstance(retries, Retry): retries = Retry.from_int(retries, redirect=redirect, default=self.retries) if release_conn is None: release_conn = response_kw.get("preload_content", True) # Check host if assert_same_host and not self.is_same_host(url): raise HostChangedError(self, url, retries) # Ensure that the URL we're connecting to is properly encoded if url.startswith("/"): url = six.ensure_str(_encode_target(url)) else: url = six.ensure_str(parsed_url.url) conn = None # Track whether `conn` needs to be released before # returning/raising/recursing. Update this variable if necessary, and # leave `release_conn` constant throughout the function. That way, if # the function recurses, the original value of `release_conn` will be # passed down into the recursive call, and its value will be respected. # # See issue #651 [1] for details. # # [1] <https://github.com/urllib3/urllib3/issues/651> release_this_conn = release_conn http_tunnel_required = connection_requires_http_tunnel( self.proxy, self.proxy_config, destination_scheme ) # Merge the proxy headers. Only done when not using HTTP CONNECT. We # have to copy the headers dict so we can safely change it without those # changes being reflected in anyone else's copy. if not http_tunnel_required: headers = headers.copy() headers.update(self.proxy_headers) # Must keep the exception bound to a separate variable or else Python 3 # complains about UnboundLocalError. err = None # Keep track of whether we cleanly exited the except block. This # ensures we do proper cleanup in finally. clean_exit = False # Rewind body position, if needed. Record current position # for future rewinds in the event of a redirect/retry. body_pos = set_file_position(body, body_pos) try: # Request a connection from the queue. timeout_obj = self._get_timeout(timeout) conn = self._get_conn(timeout=pool_timeout) conn.timeout = timeout_obj.connect_timeout is_new_proxy_conn = self.proxy is not None and not getattr( conn, "sock", None ) if is_new_proxy_conn and http_tunnel_required: self._prepare_proxy(conn) # Make the request on the httplib connection object. > httplib_response = self._make_request( conn, method, url, timeout=timeout_obj, body=body, headers=headers, chunked=chunked, ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:715: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff54053b0a0> conn = <urllib3.connection.HTTPSConnection object at 0x7ff54053b1f0> method = 'POST', url = '/user/credentials/cdis/access_token' timeout = Timeout(connect=None, read=None, total=None), chunked = False httplib_request_kw = {'body': b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwd...p, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}} timeout_obj = Timeout(connect=None, read=None, total=None) def _make_request( self, conn, method, url, timeout=_Default, chunked=False, **httplib_request_kw ): """ Perform a request on a given urllib connection object taken from our pool. :param conn: a connection from one of our connection pools :param timeout: Socket timeout in seconds for the request. This can be a float or integer, which will set the same timeout value for the socket connect and the socket read, or an instance of :class:`urllib3.util.Timeout`, which gives you more fine-grained control over your timeouts. """ self.num_requests += 1 timeout_obj = self._get_timeout(timeout) timeout_obj.start_connect() conn.timeout = Timeout.resolve_default_timeout(timeout_obj.connect_timeout) # Trigger any extra validation we need to do. try: > self._validate_conn(conn) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:404: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff54053b0a0> conn = <urllib3.connection.HTTPSConnection object at 0x7ff54053b1f0> def _validate_conn(self, conn): """ Called right before a request is made, after the socket is created. """ super(HTTPSConnectionPool, self)._validate_conn(conn) # Force connect early to allow us to validate the connection. if not getattr(conn, "sock", None): # AppEngine might not have `.sock` > conn.connect() ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:1058: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <urllib3.connection.HTTPSConnection object at 0x7ff54053b1f0> def connect(self): # Add certificate verification self.sock = conn = self._new_conn() hostname = self.host tls_in_tls = False if self._is_using_tunnel(): if self.tls_in_tls_required: self.sock = conn = self._connect_tls_proxy(hostname, conn) tls_in_tls = True # Calls self._set_hostport(), so self.host is # self._tunnel_host below. self._tunnel() # Mark this connection as not reusable self.auto_open = 0 # Override the host with the one we're requesting data from. hostname = self._tunnel_host server_hostname = hostname if self.server_hostname is not None: server_hostname = self.server_hostname is_time_off = datetime.date.today() < RECENT_DATE if is_time_off: warnings.warn( ( "System time is way off (before {0}). This will probably " "lead to SSL verification errors" ).format(RECENT_DATE), SystemTimeWarning, ) # Wrap socket using verification with the root certs in # trusted_root_certs default_ssl_context = False if self.ssl_context is None: default_ssl_context = True self.ssl_context = create_urllib3_context( ssl_version=resolve_ssl_version(self.ssl_version), cert_reqs=resolve_cert_reqs(self.cert_reqs), ) context = self.ssl_context context.verify_mode = resolve_cert_reqs(self.cert_reqs) # Try to load OS default certs if none are given. # Works well on Windows (requires Python3.4+) if ( not self.ca_certs and not self.ca_cert_dir and not self.ca_cert_data and default_ssl_context and hasattr(context, "load_default_certs") ): context.load_default_certs() > self.sock = ssl_wrap_socket( sock=conn, keyfile=self.key_file, certfile=self.cert_file, key_password=self.key_password, ca_certs=self.ca_certs, ca_cert_dir=self.ca_cert_dir, ca_cert_data=self.ca_cert_data, server_hostname=server_hostname, ssl_context=context, tls_in_tls=tls_in_tls, ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:419: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> keyfile = None, certfile = None, cert_reqs = None ca_certs = '/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/certifi/cacert.pem' server_hostname = 'gen3.abi-ctt-ctp.cloud.edu.au', ssl_version = None ciphers = None, ssl_context = <ssl.SSLContext object at 0x7ff540e2ec40> ca_cert_dir = None, key_password = None, ca_cert_data = None, tls_in_tls = False def ssl_wrap_socket( sock, keyfile=None, certfile=None, cert_reqs=None, ca_certs=None, server_hostname=None, ssl_version=None, ciphers=None, ssl_context=None, ca_cert_dir=None, key_password=None, ca_cert_data=None, tls_in_tls=False, ): """ All arguments except for server_hostname, ssl_context, and ca_cert_dir have the same meaning as they do when using :func:`ssl.wrap_socket`. :param server_hostname: When SNI is supported, the expected hostname of the certificate :param ssl_context: A pre-made :class:`SSLContext` object. If none is provided, one will be created using :func:`create_urllib3_context`. :param ciphers: A string of ciphers we wish the client to support. :param ca_cert_dir: A directory containing CA certificates in multiple separate files, as supported by OpenSSL's -CApath flag or the capath argument to SSLContext.load_verify_locations(). :param key_password: Optional password if the keyfile is encrypted. :param ca_cert_data: Optional string containing CA certificates in PEM format suitable for passing as the cadata parameter to SSLContext.load_verify_locations() :param tls_in_tls: Use SSLTransport to wrap the existing socket. """ context = ssl_context if context is None: # Note: This branch of code and all the variables in it are no longer # used by urllib3 itself. We should consider deprecating and removing # this code. context = create_urllib3_context(ssl_version, cert_reqs, ciphers=ciphers) if ca_certs or ca_cert_dir or ca_cert_data: try: context.load_verify_locations(ca_certs, ca_cert_dir, ca_cert_data) except (IOError, OSError) as e: raise SSLError(e) elif ssl_context is None and hasattr(context, "load_default_certs"): # try to load OS default certs; works well on Windows (require Python3.4+) context.load_default_certs() # Attempt to detect if we get the goofy behavior of the # keyfile being encrypted and OpenSSL asking for the # passphrase via the terminal and instead error out. if keyfile and key_password is None and _is_key_file_encrypted(keyfile): raise SSLError("Client private key is encrypted, password is required") if certfile: if key_password is None: context.load_cert_chain(certfile, keyfile) else: context.load_cert_chain(certfile, keyfile, key_password) try: if hasattr(context, "set_alpn_protocols"): context.set_alpn_protocols(ALPN_PROTOCOLS) except NotImplementedError: # Defensive: in CI, we always have set_alpn_protocols pass # If we detect server_hostname is an IP address then the SNI # extension should not be used according to RFC3546 Section 3.1 use_sni_hostname = server_hostname and not is_ipaddress(server_hostname) # SecureTransport uses server_hostname in certificate verification. send_sni = (use_sni_hostname and HAS_SNI) or ( IS_SECURETRANSPORT and server_hostname ) # Do not warn the user if server_hostname is an invalid SNI hostname. if not HAS_SNI and use_sni_hostname: warnings.warn( "An HTTPS request has been made, but the SNI (Server Name " "Indication) extension to TLS is not available on this platform. " "This may cause the server to present an incorrect TLS " "certificate, which can cause validation failures. You can upgrade to " "a newer version of Python to solve this. For more information, see " "https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html" "#ssl-warnings", SNIMissingWarning, ) if send_sni: > ssl_sock = _ssl_wrap_socket_impl( sock, context, tls_in_tls, server_hostname=server_hostname ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/ssl_.py:449: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> ssl_context = <ssl.SSLContext object at 0x7ff540e2ec40>, tls_in_tls = False server_hostname = 'gen3.abi-ctt-ctp.cloud.edu.au' def _ssl_wrap_socket_impl(sock, ssl_context, tls_in_tls, server_hostname=None): if tls_in_tls: if not SSLTransport: # Import error, ssl is not available. raise ProxySchemeUnsupported( "TLS in TLS requires support for the 'ssl' module" ) SSLTransport._validate_ssl_context_for_tls_in_tls(ssl_context) return SSLTransport(sock, ssl_context, server_hostname) if server_hostname: > return ssl_context.wrap_socket(sock, server_hostname=server_hostname) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/ssl_.py:493: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <ssl.SSLContext object at 0x7ff540e2ec40> sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> server_side = False, do_handshake_on_connect = True, suppress_ragged_eofs = True server_hostname = 'gen3.abi-ctt-ctp.cloud.edu.au', session = None def wrap_socket(self, sock, server_side=False, do_handshake_on_connect=True, suppress_ragged_eofs=True, server_hostname=None, session=None): # SSLSocket class handles server_hostname encoding before it calls # ctx._wrap_socket() > return self.sslsocket_class._create( sock=sock, server_side=server_side, do_handshake_on_connect=do_handshake_on_connect, suppress_ragged_eofs=suppress_ragged_eofs, server_hostname=server_hostname, context=self, session=session ) /usr/lib/python3.9/ssl.py:501: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ cls = <class 'ssl.SSLSocket'> sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> server_side = False, do_handshake_on_connect = True, suppress_ragged_eofs = True server_hostname = 'gen3.abi-ctt-ctp.cloud.edu.au' context = <ssl.SSLContext object at 0x7ff540e2ec40>, session = None @classmethod def _create(cls, sock, server_side=False, do_handshake_on_connect=True, suppress_ragged_eofs=True, server_hostname=None, context=None, session=None): if sock.getsockopt(SOL_SOCKET, SO_TYPE) != SOCK_STREAM: raise NotImplementedError("only stream sockets are supported") if server_side: if server_hostname: raise ValueError("server_hostname can only be specified " "in client mode") if session is not None: raise ValueError("session can only be specified in " "client mode") if context.check_hostname and not server_hostname: raise ValueError("check_hostname requires server_hostname") kwargs = dict( family=sock.family, type=sock.type, proto=sock.proto, fileno=sock.fileno() ) self = cls.__new__(cls, **kwargs) super(SSLSocket, self).__init__(**kwargs) sock_timeout = sock.gettimeout() sock.detach() self._context = context self._session = session self._closed = False self._sslobj = None self.server_side = server_side self.server_hostname = context._encode_hostname(server_hostname) self.do_handshake_on_connect = do_handshake_on_connect self.suppress_ragged_eofs = suppress_ragged_eofs # See if we are connected try: self.getpeername() except OSError as e: if e.errno != errno.ENOTCONN: raise connected = False blocking = self.getblocking() self.setblocking(False) try: # We are not connected so this is not supposed to block, but # testing revealed otherwise on macOS and Windows so we do # the non-blocking dance regardless. Our raise when any data # is found means consuming the data is harmless. notconn_pre_handshake_data = self.recv(1) except OSError as e: # EINVAL occurs for recv(1) on non-connected on unix sockets. if e.errno not in (errno.ENOTCONN, errno.EINVAL): raise notconn_pre_handshake_data = b'' self.setblocking(blocking) if notconn_pre_handshake_data: # This prevents pending data sent to the socket before it was # closed from escaping to the caller who could otherwise # presume it came through a successful TLS connection. reason = "Closed before TLS handshake with data in recv buffer." notconn_pre_handshake_data_error = SSLError(e.errno, reason) # Add the SSLError attributes that _ssl.c always adds. notconn_pre_handshake_data_error.reason = reason notconn_pre_handshake_data_error.library = None try: self.close() except OSError: pass try: raise notconn_pre_handshake_data_error finally: # Explicitly break the reference cycle. notconn_pre_handshake_data_error = None else: connected = True self.settimeout(sock_timeout) # Must come after setblocking() calls. self._connected = connected if connected: # create the SSL object try: self._sslobj = self._context._wrap_socket( self, server_side, self.server_hostname, owner=self, session=self._session, ) if do_handshake_on_connect: timeout = self.gettimeout() if timeout == 0.0: # non-blocking raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets") > self.do_handshake() /usr/lib/python3.9/ssl.py:1074: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <ssl.SSLSocket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> block = False @_sslcopydoc def do_handshake(self, block=False): self._check_connected() timeout = self.gettimeout() try: if timeout == 0.0 and block: self.settimeout(None) > self._sslobj.do_handshake() E ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133) /usr/lib/python3.9/ssl.py:1343: SSLCertVerificationError During handling of the above exception, another exception occurred: self = <requests.adapters.HTTPAdapter object at 0x7ff54053be20> request = <PreparedRequest [POST]>, stream = False timeout = Timeout(connect=None, read=None, total=None), verify = True cert = None, proxies = OrderedDict() def send( self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None ): """Sends PreparedRequest object. Returns Response object. :param request: The :class:`PreparedRequest <PreparedRequest>` being sent. :param stream: (optional) Whether to stream the request content. :param timeout: (optional) How long to wait for the server to send data before giving up, as a float, or a :ref:`(connect timeout, read timeout) <timeouts>` tuple. :type timeout: float or tuple or urllib3 Timeout object :param verify: (optional) Either a boolean, in which case it controls whether we verify the server's TLS certificate, or a string, in which case it must be a path to a CA bundle to use :param cert: (optional) Any user-provided SSL certificate to be trusted. :param proxies: (optional) The proxies dictionary to apply to the request. :rtype: requests.Response """ try: conn = self.get_connection(request.url, proxies) except LocationValueError as e: raise InvalidURL(e, request=request) self.cert_verify(conn, request.url, verify, cert) url = self.request_url(request, proxies) self.add_headers( request, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies, ) chunked = not (request.body is None or "Content-Length" in request.headers) if isinstance(timeout, tuple): try: connect, read = timeout timeout = TimeoutSauce(connect=connect, read=read) except ValueError: raise ValueError( f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, " f"or a single float to set both timeouts to the same value." ) elif isinstance(timeout, TimeoutSauce): pass else: timeout = TimeoutSauce(connect=timeout, read=timeout) try: > resp = conn.urlopen( method=request.method, url=url, body=request.body, headers=request.headers, redirect=False, assert_same_host=False, preload_content=False, decode_content=False, retries=self.max_retries, timeout=timeout, chunked=chunked, ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:486: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff54053b0a0> method = 'POST', url = '/user/credentials/cdis/access_token' body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...NPva0i6TdvsONgWYV1N3mET3cX78Px5n6SH84sTMCLqWr5EOzw4ImhcQ2kOYfMOqKA", "key_id": "92c93027-24d6-4240-825c-30b9395d78cf"}' headers = {'User-Agent': 'python-requests/2.31.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'} retries = Retry(total=0, connect=None, read=False, redirect=None, status=None) redirect = False, assert_same_host = False timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None release_conn = False, chunked = False, body_pos = None response_kw = {'decode_content': False, 'preload_content': False} parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None) destination_scheme = None, conn = None, release_this_conn = True http_tunnel_required = False, err = None, clean_exit = False def urlopen( self, method, url, body=None, headers=None, retries=None, redirect=True, assert_same_host=True, timeout=_Default, pool_timeout=None, release_conn=None, chunked=False, body_pos=None, **response_kw ): """ Get a connection from the pool and perform an HTTP request. This is the lowest level call for making a request, so you'll need to specify all the raw details. .. note:: More commonly, it's appropriate to use a convenience method provided by :class:`.RequestMethods`, such as :meth:`request`. .. note:: `release_conn` will only behave as expected if `preload_content=False` because we want to make `preload_content=False` the default behaviour someday soon without breaking backwards compatibility. :param method: HTTP request method (such as GET, POST, PUT, etc.) :param url: The URL to perform the request on. :param body: Data to send in the request body, either :class:`str`, :class:`bytes`, an iterable of :class:`str`/:class:`bytes`, or a file-like object. :param headers: Dictionary of custom headers to send, such as User-Agent, If-None-Match, etc. If None, pool headers are used. If provided, these headers completely replace any pool-specific headers. :param retries: Configure the number of retries to allow before raising a :class:`~urllib3.exceptions.MaxRetryError` exception. Pass ``None`` to retry until you receive a response. Pass a :class:`~urllib3.util.retry.Retry` object for fine-grained control over different types of retries. Pass an integer number to retry connection errors that many times, but no other types of errors. Pass zero to never retry. If ``False``, then retries are disabled and any exception is raised immediately. Also, instead of raising a MaxRetryError on redirects, the redirect response will be returned. :type retries: :class:`~urllib3.util.retry.Retry`, False, or an int. :param redirect: If True, automatically handle redirects (status codes 301, 302, 303, 307, 308). Each redirect counts as a retry. Disabling retries will disable redirect, too. :param assert_same_host: If ``True``, will make sure that the host of the pool requests is consistent else will raise HostChangedError. When ``False``, you can use the pool on an HTTP proxy and request foreign hosts. :param timeout: If specified, overrides the default timeout for this one request. It may be a float (in seconds) or an instance of :class:`urllib3.util.Timeout`. :param pool_timeout: If set and the pool is set to block=True, then this method will block for ``pool_timeout`` seconds and raise EmptyPoolError if no connection is available within the time period. :param release_conn: If False, then the urlopen call will not release the connection back into the pool once a response is received (but will release if you read the entire contents of the response such as when `preload_content=True`). This is useful if you're not preloading the response's content immediately. You will need to call ``r.release_conn()`` on the response ``r`` to return the connection back into the pool. If None, it takes the value of ``response_kw.get('preload_content', True)``. :param chunked: If True, urllib3 will send the body using chunked transfer encoding. Otherwise, urllib3 will send the body using the standard content-length form. Defaults to False. :param int body_pos: Position to seek to in file-like body in the event of a retry or redirect. Typically this won't need to be set because urllib3 will auto-populate the value when needed. :param \\**response_kw: Additional parameters are passed to :meth:`urllib3.response.HTTPResponse.from_httplib` """ parsed_url = parse_url(url) destination_scheme = parsed_url.scheme if headers is None: headers = self.headers if not isinstance(retries, Retry): retries = Retry.from_int(retries, redirect=redirect, default=self.retries) if release_conn is None: release_conn = response_kw.get("preload_content", True) # Check host if assert_same_host and not self.is_same_host(url): raise HostChangedError(self, url, retries) # Ensure that the URL we're connecting to is properly encoded if url.startswith("/"): url = six.ensure_str(_encode_target(url)) else: url = six.ensure_str(parsed_url.url) conn = None # Track whether `conn` needs to be released before # returning/raising/recursing. Update this variable if necessary, and # leave `release_conn` constant throughout the function. That way, if # the function recurses, the original value of `release_conn` will be # passed down into the recursive call, and its value will be respected. # # See issue #651 [1] for details. # # [1] <https://github.com/urllib3/urllib3/issues/651> release_this_conn = release_conn http_tunnel_required = connection_requires_http_tunnel( self.proxy, self.proxy_config, destination_scheme ) # Merge the proxy headers. Only done when not using HTTP CONNECT. We # have to copy the headers dict so we can safely change it without those # changes being reflected in anyone else's copy. if not http_tunnel_required: headers = headers.copy() headers.update(self.proxy_headers) # Must keep the exception bound to a separate variable or else Python 3 # complains about UnboundLocalError. err = None # Keep track of whether we cleanly exited the except block. This # ensures we do proper cleanup in finally. clean_exit = False # Rewind body position, if needed. Record current position # for future rewinds in the event of a redirect/retry. body_pos = set_file_position(body, body_pos) try: # Request a connection from the queue. timeout_obj = self._get_timeout(timeout) conn = self._get_conn(timeout=pool_timeout) conn.timeout = timeout_obj.connect_timeout is_new_proxy_conn = self.proxy is not None and not getattr( conn, "sock", None ) if is_new_proxy_conn and http_tunnel_required: self._prepare_proxy(conn) # Make the request on the httplib connection object. httplib_response = self._make_request( conn, method, url, timeout=timeout_obj, body=body, headers=headers, chunked=chunked, ) # If we're going to release the connection in ``finally:``, then # the response doesn't need to know about the connection. Otherwise # it will also try to release it and we'll have a double-release # mess. response_conn = conn if not release_conn else None # Pass method to Response for length checking response_kw["request_method"] = method # Import httplib's response into our own wrapper object response = self.ResponseCls.from_httplib( httplib_response, pool=self, connection=response_conn, retries=retries, **response_kw ) # Everything went great! clean_exit = True except EmptyPoolError: # Didn't get a connection from the pool, no need to clean up clean_exit = True release_this_conn = False raise except ( TimeoutError, HTTPException, SocketError, ProtocolError, BaseSSLError, SSLError, CertificateError, ) as e: # Discard the connection for these exceptions. It will be # replaced during the next _get_conn() call. clean_exit = False def _is_ssl_error_message_from_http_proxy(ssl_error): # We're trying to detect the message 'WRONG_VERSION_NUMBER' but # SSLErrors are kinda all over the place when it comes to the message, # so we try to cover our bases here! message = " ".join(re.split("[^a-z]", str(ssl_error).lower())) return ( "wrong version number" in message or "unknown protocol" in message ) # Try to detect a common user error with proxies which is to # set an HTTP proxy to be HTTPS when it should be 'http://' # (ie {'http': 'http://proxy', 'https': 'https://proxy'}) # Instead we add a nice error message and point to a URL. if ( isinstance(e, BaseSSLError) and self.proxy and _is_ssl_error_message_from_http_proxy(e) and conn.proxy and conn.proxy.scheme == "https" ): e = ProxyError( "Your proxy appears to only use HTTP and not HTTPS, " "try changing your proxy URL to be HTTP. See: " "https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html" "#https-proxy-error-http-proxy", SSLError(e), ) elif isinstance(e, (BaseSSLError, CertificateError)): e = SSLError(e) elif isinstance(e, (SocketError, NewConnectionError)) and self.proxy: e = ProxyError("Cannot connect to proxy.", e) elif isinstance(e, (SocketError, HTTPException)): e = ProtocolError("Connection aborted.", e) > retries = retries.increment( method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:799: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = Retry(total=0, connect=None, read=False, redirect=None, status=None) method = 'POST', url = '/user/credentials/cdis/access_token', response = None error = SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133)')) _pool = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff54053b0a0> _stacktrace = <traceback object at 0x7ff541f85380> def increment( self, method=None, url=None, response=None, error=None, _pool=None, _stacktrace=None, ): """Return a new Retry object with incremented retry counters. :param response: A response object, or None, if the server did not return a response. :type response: :class:`~urllib3.response.HTTPResponse` :param Exception error: An error encountered during the request, or None if the response was received successfully. :return: A new ``Retry`` object. """ if self.total is False and error: # Disabled, indicate to re-raise the error. raise six.reraise(type(error), error, _stacktrace) total = self.total if total is not None: total -= 1 connect = self.connect read = self.read redirect = self.redirect status_count = self.status other = self.other cause = "unknown" status = None redirect_location = None if error and self._is_connection_error(error): # Connect retry? if connect is False: raise six.reraise(type(error), error, _stacktrace) elif connect is not None: connect -= 1 elif error and self._is_read_error(error): # Read retry? if read is False or not self._is_method_retryable(method): raise six.reraise(type(error), error, _stacktrace) elif read is not None: read -= 1 elif error: # Other retry? if other is not None: other -= 1 elif response and response.get_redirect_location(): # Redirect retry? if redirect is not None: redirect -= 1 cause = "too many redirects" redirect_location = response.get_redirect_location() status = response.status else: # Incrementing because of a server error like a 500 in # status_forcelist and the given method is in the allowed_methods cause = ResponseError.GENERIC_ERROR if response and response.status: if status_count is not None: status_count -= 1 cause = ResponseError.SPECIFIC_ERROR.format(status_code=response.status) status = response.status history = self.history + ( RequestHistory(method, url, error, status, redirect_location), ) new_retry = self.new( total=total, connect=connect, read=read, redirect=redirect, status=status_count, other=other, history=history, ) if new_retry.is_exhausted(): > raise MaxRetryError(_pool, url, error or ResponseError(cause)) E urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133)'))) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/retry.py:592: MaxRetryError During handling of the above exception, another exception occurred: client = <starlette.testclient.TestClient object at 0x7ff5404b41c0> token = {'access_token': 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZGVudGl0eSI6InB1YmxpYyIsInNjb3BlIjpbImRlbW8xLTEyTCJdLCJleHBpcmUiOiJOb25lIn0.doEgWg8P2u6dGhajC1CBwJHS3TtbJh4L_vhMfK75uJQ'} def test_experiment_node(client, token): UUID = "22c4459b-5f4f-4e62-abd2-2aa205fe838b" > response = client.get( f"/record/{UUID}", headers={"Authorization": f"Bearer {token['access_token']}"}, ) tests/test_data/test_gen3_data.py:31: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:602: in get return self.request("GET", url, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:476: in request return super().request( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:589: in request resp = self.send(prep, **send_kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:703: in send r = adapter.send(request, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:270: in send raise exc ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:267: in send portal.call(self.app, scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:283: in call return cast(T_Retval, self.start_task_soon(func, *args).result()) /usr/lib/python3.9/concurrent/futures/_base.py:446: in result return self.__get_result() /usr/lib/python3.9/concurrent/futures/_base.py:391: in __get_result raise self._exception ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:219: in _call_func retval = await retval ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/applications.py:269: in __call__ await super().__call__(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/applications.py:124: in __call__ await self.middleware_stack(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:184: in __call__ raise exc ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:162: in __call__ await self.app(scope, receive, _send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/cors.py:84: in __call__ await self.app(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:93: in __call__ raise exc ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:82: in __call__ await self.app(scope, receive, sender) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:21: in __call__ raise e ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:18: in __call__ await self.app(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:670: in __call__ await route.handle(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:266: in handle await self.app(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:65: in app response = await func(request) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:227: in app raw_response = await run_endpoint_function( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:160: in run_endpoint_function return await dependant.call(**values) app/main.py:318: in get_gen3_record record = connection["gen3"].export_record(program, project, uuid, "json") ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/submission.py:348: in export_record output = requests.get(api_url, auth=self._auth_provider).text ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:73: in get return request("get", url, params=params, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request return session.request(method=method, url=url, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:575: in request prep = self.prepare_request(req) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:486: in prepare_request p.prepare( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:372: in prepare self.prepare_auth(auth, url) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:603: in prepare_auth r = auth(self) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:325: in __call__ request.headers["Authorization"] = self._get_auth_value() ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:446: in _get_auth_value return "bearer " + self.get_access_token() ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:433: in get_access_token return self.refresh_access_token( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:367: in refresh_access_token self._access_token = get_access_token_with_key(self._refresh_token) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:75: in get_access_token_with_key resp = requests.post(auth_url, json=api_key) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:115: in post return request("post", url, data=data, json=json, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request return session.request(method=method, url=url, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:589: in request resp = self.send(prep, **send_kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:703: in send r = adapter.send(request, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <requests.adapters.HTTPAdapter object at 0x7ff54053be20> request = <PreparedRequest [POST]>, stream = False timeout = Timeout(connect=None, read=None, total=None), verify = True cert = None, proxies = OrderedDict() def send( self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None ): """Sends PreparedRequest object. Returns Response object. :param request: The :class:`PreparedRequest <PreparedRequest>` being sent. :param stream: (optional) Whether to stream the request content. :param timeout: (optional) How long to wait for the server to send data before giving up, as a float, or a :ref:`(connect timeout, read timeout) <timeouts>` tuple. :type timeout: float or tuple or urllib3 Timeout object :param verify: (optional) Either a boolean, in which case it controls whether we verify the server's TLS certificate, or a string, in which case it must be a path to a CA bundle to use :param cert: (optional) Any user-provided SSL certificate to be trusted. :param proxies: (optional) The proxies dictionary to apply to the request. :rtype: requests.Response """ try: conn = self.get_connection(request.url, proxies) except LocationValueError as e: raise InvalidURL(e, request=request) self.cert_verify(conn, request.url, verify, cert) url = self.request_url(request, proxies) self.add_headers( request, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies, ) chunked = not (request.body is None or "Content-Length" in request.headers) if isinstance(timeout, tuple): try: connect, read = timeout timeout = TimeoutSauce(connect=connect, read=read) except ValueError: raise ValueError( f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, " f"or a single float to set both timeouts to the same value." ) elif isinstance(timeout, TimeoutSauce): pass else: timeout = TimeoutSauce(connect=timeout, read=timeout) try: resp = conn.urlopen( method=request.method, url=url, body=request.body, headers=request.headers, redirect=False, assert_same_host=False, preload_content=False, decode_content=False, retries=self.max_retries, timeout=timeout, chunked=chunked, ) except (ProtocolError, OSError) as err: raise ConnectionError(err, request=request) except MaxRetryError as e: if isinstance(e.reason, ConnectTimeoutError): # TODO: Remove this in 3.0.0: see #2811 if not isinstance(e.reason, NewConnectionError): raise ConnectTimeout(e, request=request) if isinstance(e.reason, ResponseError): raise RetryError(e, request=request) if isinstance(e.reason, _ProxyError): raise ProxyError(e, request=request) if isinstance(e.reason, _SSLError): # This branch is for urllib3 v1.22 and later. > raise SSLError(e, request=request) E requests.exceptions.SSLError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133)'))) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:517: SSLError ------------------------------ Captured log setup ------------------------------ ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. INFO app.main:main.py:166 {'gen3': False, 'irods': True, 'orthanc': True} WARNING app.main:main.py:185 Failed to update default filter. ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. ------------------------------ Captured log call ------------------------------- ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. ________________________ test_dataset_description_node _________________________ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff54084fbe0> method = 'POST', url = '/user/credentials/cdis/access_token' body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...NPva0i6TdvsONgWYV1N3mET3cX78Px5n6SH84sTMCLqWr5EOzw4ImhcQ2kOYfMOqKA", "key_id": "92c93027-24d6-4240-825c-30b9395d78cf"}' headers = {'User-Agent': 'python-requests/2.31.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'} retries = Retry(total=0, connect=None, read=False, redirect=None, status=None) redirect = False, assert_same_host = False timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None release_conn = False, chunked = False, body_pos = None response_kw = {'decode_content': False, 'preload_content': False} parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None) destination_scheme = None, conn = None, release_this_conn = True http_tunnel_required = False, err = None, clean_exit = False def urlopen( self, method, url, body=None, headers=None, retries=None, redirect=True, assert_same_host=True, timeout=_Default, pool_timeout=None, release_conn=None, chunked=False, body_pos=None, **response_kw ): """ Get a connection from the pool and perform an HTTP request. This is the lowest level call for making a request, so you'll need to specify all the raw details. .. note:: More commonly, it's appropriate to use a convenience method provided by :class:`.RequestMethods`, such as :meth:`request`. .. note:: `release_conn` will only behave as expected if `preload_content=False` because we want to make `preload_content=False` the default behaviour someday soon without breaking backwards compatibility. :param method: HTTP request method (such as GET, POST, PUT, etc.) :param url: The URL to perform the request on. :param body: Data to send in the request body, either :class:`str`, :class:`bytes`, an iterable of :class:`str`/:class:`bytes`, or a file-like object. :param headers: Dictionary of custom headers to send, such as User-Agent, If-None-Match, etc. If None, pool headers are used. If provided, these headers completely replace any pool-specific headers. :param retries: Configure the number of retries to allow before raising a :class:`~urllib3.exceptions.MaxRetryError` exception. Pass ``None`` to retry until you receive a response. Pass a :class:`~urllib3.util.retry.Retry` object for fine-grained control over different types of retries. Pass an integer number to retry connection errors that many times, but no other types of errors. Pass zero to never retry. If ``False``, then retries are disabled and any exception is raised immediately. Also, instead of raising a MaxRetryError on redirects, the redirect response will be returned. :type retries: :class:`~urllib3.util.retry.Retry`, False, or an int. :param redirect: If True, automatically handle redirects (status codes 301, 302, 303, 307, 308). Each redirect counts as a retry. Disabling retries will disable redirect, too. :param assert_same_host: If ``True``, will make sure that the host of the pool requests is consistent else will raise HostChangedError. When ``False``, you can use the pool on an HTTP proxy and request foreign hosts. :param timeout: If specified, overrides the default timeout for this one request. It may be a float (in seconds) or an instance of :class:`urllib3.util.Timeout`. :param pool_timeout: If set and the pool is set to block=True, then this method will block for ``pool_timeout`` seconds and raise EmptyPoolError if no connection is available within the time period. :param release_conn: If False, then the urlopen call will not release the connection back into the pool once a response is received (but will release if you read the entire contents of the response such as when `preload_content=True`). This is useful if you're not preloading the response's content immediately. You will need to call ``r.release_conn()`` on the response ``r`` to return the connection back into the pool. If None, it takes the value of ``response_kw.get('preload_content', True)``. :param chunked: If True, urllib3 will send the body using chunked transfer encoding. Otherwise, urllib3 will send the body using the standard content-length form. Defaults to False. :param int body_pos: Position to seek to in file-like body in the event of a retry or redirect. Typically this won't need to be set because urllib3 will auto-populate the value when needed. :param \\**response_kw: Additional parameters are passed to :meth:`urllib3.response.HTTPResponse.from_httplib` """ parsed_url = parse_url(url) destination_scheme = parsed_url.scheme if headers is None: headers = self.headers if not isinstance(retries, Retry): retries = Retry.from_int(retries, redirect=redirect, default=self.retries) if release_conn is None: release_conn = response_kw.get("preload_content", True) # Check host if assert_same_host and not self.is_same_host(url): raise HostChangedError(self, url, retries) # Ensure that the URL we're connecting to is properly encoded if url.startswith("/"): url = six.ensure_str(_encode_target(url)) else: url = six.ensure_str(parsed_url.url) conn = None # Track whether `conn` needs to be released before # returning/raising/recursing. Update this variable if necessary, and # leave `release_conn` constant throughout the function. That way, if # the function recurses, the original value of `release_conn` will be # passed down into the recursive call, and its value will be respected. # # See issue #651 [1] for details. # # [1] <https://github.com/urllib3/urllib3/issues/651> release_this_conn = release_conn http_tunnel_required = connection_requires_http_tunnel( self.proxy, self.proxy_config, destination_scheme ) # Merge the proxy headers. Only done when not using HTTP CONNECT. We # have to copy the headers dict so we can safely change it without those # changes being reflected in anyone else's copy. if not http_tunnel_required: headers = headers.copy() headers.update(self.proxy_headers) # Must keep the exception bound to a separate variable or else Python 3 # complains about UnboundLocalError. err = None # Keep track of whether we cleanly exited the except block. This # ensures we do proper cleanup in finally. clean_exit = False # Rewind body position, if needed. Record current position # for future rewinds in the event of a redirect/retry. body_pos = set_file_position(body, body_pos) try: # Request a connection from the queue. timeout_obj = self._get_timeout(timeout) conn = self._get_conn(timeout=pool_timeout) conn.timeout = timeout_obj.connect_timeout is_new_proxy_conn = self.proxy is not None and not getattr( conn, "sock", None ) if is_new_proxy_conn and http_tunnel_required: self._prepare_proxy(conn) # Make the request on the httplib connection object. > httplib_response = self._make_request( conn, method, url, timeout=timeout_obj, body=body, headers=headers, chunked=chunked, ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:715: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff54084fbe0> conn = <urllib3.connection.HTTPSConnection object at 0x7ff54084f7f0> method = 'POST', url = '/user/credentials/cdis/access_token' timeout = Timeout(connect=None, read=None, total=None), chunked = False httplib_request_kw = {'body': b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwd...p, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}} timeout_obj = Timeout(connect=None, read=None, total=None) def _make_request( self, conn, method, url, timeout=_Default, chunked=False, **httplib_request_kw ): """ Perform a request on a given urllib connection object taken from our pool. :param conn: a connection from one of our connection pools :param timeout: Socket timeout in seconds for the request. This can be a float or integer, which will set the same timeout value for the socket connect and the socket read, or an instance of :class:`urllib3.util.Timeout`, which gives you more fine-grained control over your timeouts. """ self.num_requests += 1 timeout_obj = self._get_timeout(timeout) timeout_obj.start_connect() conn.timeout = Timeout.resolve_default_timeout(timeout_obj.connect_timeout) # Trigger any extra validation we need to do. try: > self._validate_conn(conn) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:404: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff54084fbe0> conn = <urllib3.connection.HTTPSConnection object at 0x7ff54084f7f0> def _validate_conn(self, conn): """ Called right before a request is made, after the socket is created. """ super(HTTPSConnectionPool, self)._validate_conn(conn) # Force connect early to allow us to validate the connection. if not getattr(conn, "sock", None): # AppEngine might not have `.sock` > conn.connect() ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:1058: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <urllib3.connection.HTTPSConnection object at 0x7ff54084f7f0> def connect(self): # Add certificate verification self.sock = conn = self._new_conn() hostname = self.host tls_in_tls = False if self._is_using_tunnel(): if self.tls_in_tls_required: self.sock = conn = self._connect_tls_proxy(hostname, conn) tls_in_tls = True # Calls self._set_hostport(), so self.host is # self._tunnel_host below. self._tunnel() # Mark this connection as not reusable self.auto_open = 0 # Override the host with the one we're requesting data from. hostname = self._tunnel_host server_hostname = hostname if self.server_hostname is not None: server_hostname = self.server_hostname is_time_off = datetime.date.today() < RECENT_DATE if is_time_off: warnings.warn( ( "System time is way off (before {0}). This will probably " "lead to SSL verification errors" ).format(RECENT_DATE), SystemTimeWarning, ) # Wrap socket using verification with the root certs in # trusted_root_certs default_ssl_context = False if self.ssl_context is None: default_ssl_context = True self.ssl_context = create_urllib3_context( ssl_version=resolve_ssl_version(self.ssl_version), cert_reqs=resolve_cert_reqs(self.cert_reqs), ) context = self.ssl_context context.verify_mode = resolve_cert_reqs(self.cert_reqs) # Try to load OS default certs if none are given. # Works well on Windows (requires Python3.4+) if ( not self.ca_certs and not self.ca_cert_dir and not self.ca_cert_data and default_ssl_context and hasattr(context, "load_default_certs") ): context.load_default_certs() > self.sock = ssl_wrap_socket( sock=conn, keyfile=self.key_file, certfile=self.cert_file, key_password=self.key_password, ca_certs=self.ca_certs, ca_cert_dir=self.ca_cert_dir, ca_cert_data=self.ca_cert_data, server_hostname=server_hostname, ssl_context=context, tls_in_tls=tls_in_tls, ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:419: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> keyfile = None, certfile = None, cert_reqs = None ca_certs = '/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/certifi/cacert.pem' server_hostname = 'gen3.abi-ctt-ctp.cloud.edu.au', ssl_version = None ciphers = None, ssl_context = <ssl.SSLContext object at 0x7ff5405ed440> ca_cert_dir = None, key_password = None, ca_cert_data = None, tls_in_tls = False def ssl_wrap_socket( sock, keyfile=None, certfile=None, cert_reqs=None, ca_certs=None, server_hostname=None, ssl_version=None, ciphers=None, ssl_context=None, ca_cert_dir=None, key_password=None, ca_cert_data=None, tls_in_tls=False, ): """ All arguments except for server_hostname, ssl_context, and ca_cert_dir have the same meaning as they do when using :func:`ssl.wrap_socket`. :param server_hostname: When SNI is supported, the expected hostname of the certificate :param ssl_context: A pre-made :class:`SSLContext` object. If none is provided, one will be created using :func:`create_urllib3_context`. :param ciphers: A string of ciphers we wish the client to support. :param ca_cert_dir: A directory containing CA certificates in multiple separate files, as supported by OpenSSL's -CApath flag or the capath argument to SSLContext.load_verify_locations(). :param key_password: Optional password if the keyfile is encrypted. :param ca_cert_data: Optional string containing CA certificates in PEM format suitable for passing as the cadata parameter to SSLContext.load_verify_locations() :param tls_in_tls: Use SSLTransport to wrap the existing socket. """ context = ssl_context if context is None: # Note: This branch of code and all the variables in it are no longer # used by urllib3 itself. We should consider deprecating and removing # this code. context = create_urllib3_context(ssl_version, cert_reqs, ciphers=ciphers) if ca_certs or ca_cert_dir or ca_cert_data: try: context.load_verify_locations(ca_certs, ca_cert_dir, ca_cert_data) except (IOError, OSError) as e: raise SSLError(e) elif ssl_context is None and hasattr(context, "load_default_certs"): # try to load OS default certs; works well on Windows (require Python3.4+) context.load_default_certs() # Attempt to detect if we get the goofy behavior of the # keyfile being encrypted and OpenSSL asking for the # passphrase via the terminal and instead error out. if keyfile and key_password is None and _is_key_file_encrypted(keyfile): raise SSLError("Client private key is encrypted, password is required") if certfile: if key_password is None: context.load_cert_chain(certfile, keyfile) else: context.load_cert_chain(certfile, keyfile, key_password) try: if hasattr(context, "set_alpn_protocols"): context.set_alpn_protocols(ALPN_PROTOCOLS) except NotImplementedError: # Defensive: in CI, we always have set_alpn_protocols pass # If we detect server_hostname is an IP address then the SNI # extension should not be used according to RFC3546 Section 3.1 use_sni_hostname = server_hostname and not is_ipaddress(server_hostname) # SecureTransport uses server_hostname in certificate verification. send_sni = (use_sni_hostname and HAS_SNI) or ( IS_SECURETRANSPORT and server_hostname ) # Do not warn the user if server_hostname is an invalid SNI hostname. if not HAS_SNI and use_sni_hostname: warnings.warn( "An HTTPS request has been made, but the SNI (Server Name " "Indication) extension to TLS is not available on this platform. " "This may cause the server to present an incorrect TLS " "certificate, which can cause validation failures. You can upgrade to " "a newer version of Python to solve this. For more information, see " "https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html" "#ssl-warnings", SNIMissingWarning, ) if send_sni: > ssl_sock = _ssl_wrap_socket_impl( sock, context, tls_in_tls, server_hostname=server_hostname ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/ssl_.py:449: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> ssl_context = <ssl.SSLContext object at 0x7ff5405ed440>, tls_in_tls = False server_hostname = 'gen3.abi-ctt-ctp.cloud.edu.au' def _ssl_wrap_socket_impl(sock, ssl_context, tls_in_tls, server_hostname=None): if tls_in_tls: if not SSLTransport: # Import error, ssl is not available. raise ProxySchemeUnsupported( "TLS in TLS requires support for the 'ssl' module" ) SSLTransport._validate_ssl_context_for_tls_in_tls(ssl_context) return SSLTransport(sock, ssl_context, server_hostname) if server_hostname: > return ssl_context.wrap_socket(sock, server_hostname=server_hostname) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/ssl_.py:493: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <ssl.SSLContext object at 0x7ff5405ed440> sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> server_side = False, do_handshake_on_connect = True, suppress_ragged_eofs = True server_hostname = 'gen3.abi-ctt-ctp.cloud.edu.au', session = None def wrap_socket(self, sock, server_side=False, do_handshake_on_connect=True, suppress_ragged_eofs=True, server_hostname=None, session=None): # SSLSocket class handles server_hostname encoding before it calls # ctx._wrap_socket() > return self.sslsocket_class._create( sock=sock, server_side=server_side, do_handshake_on_connect=do_handshake_on_connect, suppress_ragged_eofs=suppress_ragged_eofs, server_hostname=server_hostname, context=self, session=session ) /usr/lib/python3.9/ssl.py:501: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ cls = <class 'ssl.SSLSocket'> sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> server_side = False, do_handshake_on_connect = True, suppress_ragged_eofs = True server_hostname = 'gen3.abi-ctt-ctp.cloud.edu.au' context = <ssl.SSLContext object at 0x7ff5405ed440>, session = None @classmethod def _create(cls, sock, server_side=False, do_handshake_on_connect=True, suppress_ragged_eofs=True, server_hostname=None, context=None, session=None): if sock.getsockopt(SOL_SOCKET, SO_TYPE) != SOCK_STREAM: raise NotImplementedError("only stream sockets are supported") if server_side: if server_hostname: raise ValueError("server_hostname can only be specified " "in client mode") if session is not None: raise ValueError("session can only be specified in " "client mode") if context.check_hostname and not server_hostname: raise ValueError("check_hostname requires server_hostname") kwargs = dict( family=sock.family, type=sock.type, proto=sock.proto, fileno=sock.fileno() ) self = cls.__new__(cls, **kwargs) super(SSLSocket, self).__init__(**kwargs) sock_timeout = sock.gettimeout() sock.detach() self._context = context self._session = session self._closed = False self._sslobj = None self.server_side = server_side self.server_hostname = context._encode_hostname(server_hostname) self.do_handshake_on_connect = do_handshake_on_connect self.suppress_ragged_eofs = suppress_ragged_eofs # See if we are connected try: self.getpeername() except OSError as e: if e.errno != errno.ENOTCONN: raise connected = False blocking = self.getblocking() self.setblocking(False) try: # We are not connected so this is not supposed to block, but # testing revealed otherwise on macOS and Windows so we do # the non-blocking dance regardless. Our raise when any data # is found means consuming the data is harmless. notconn_pre_handshake_data = self.recv(1) except OSError as e: # EINVAL occurs for recv(1) on non-connected on unix sockets. if e.errno not in (errno.ENOTCONN, errno.EINVAL): raise notconn_pre_handshake_data = b'' self.setblocking(blocking) if notconn_pre_handshake_data: # This prevents pending data sent to the socket before it was # closed from escaping to the caller who could otherwise # presume it came through a successful TLS connection. reason = "Closed before TLS handshake with data in recv buffer." notconn_pre_handshake_data_error = SSLError(e.errno, reason) # Add the SSLError attributes that _ssl.c always adds. notconn_pre_handshake_data_error.reason = reason notconn_pre_handshake_data_error.library = None try: self.close() except OSError: pass try: raise notconn_pre_handshake_data_error finally: # Explicitly break the reference cycle. notconn_pre_handshake_data_error = None else: connected = True self.settimeout(sock_timeout) # Must come after setblocking() calls. self._connected = connected if connected: # create the SSL object try: self._sslobj = self._context._wrap_socket( self, server_side, self.server_hostname, owner=self, session=self._session, ) if do_handshake_on_connect: timeout = self.gettimeout() if timeout == 0.0: # non-blocking raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets") > self.do_handshake() /usr/lib/python3.9/ssl.py:1074: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <ssl.SSLSocket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> block = False @_sslcopydoc def do_handshake(self, block=False): self._check_connected() timeout = self.gettimeout() try: if timeout == 0.0 and block: self.settimeout(None) > self._sslobj.do_handshake() E ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133) /usr/lib/python3.9/ssl.py:1343: SSLCertVerificationError During handling of the above exception, another exception occurred: self = <requests.adapters.HTTPAdapter object at 0x7ff54084f790> request = <PreparedRequest [POST]>, stream = False timeout = Timeout(connect=None, read=None, total=None), verify = True cert = None, proxies = OrderedDict() def send( self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None ): """Sends PreparedRequest object. Returns Response object. :param request: The :class:`PreparedRequest <PreparedRequest>` being sent. :param stream: (optional) Whether to stream the request content. :param timeout: (optional) How long to wait for the server to send data before giving up, as a float, or a :ref:`(connect timeout, read timeout) <timeouts>` tuple. :type timeout: float or tuple or urllib3 Timeout object :param verify: (optional) Either a boolean, in which case it controls whether we verify the server's TLS certificate, or a string, in which case it must be a path to a CA bundle to use :param cert: (optional) Any user-provided SSL certificate to be trusted. :param proxies: (optional) The proxies dictionary to apply to the request. :rtype: requests.Response """ try: conn = self.get_connection(request.url, proxies) except LocationValueError as e: raise InvalidURL(e, request=request) self.cert_verify(conn, request.url, verify, cert) url = self.request_url(request, proxies) self.add_headers( request, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies, ) chunked = not (request.body is None or "Content-Length" in request.headers) if isinstance(timeout, tuple): try: connect, read = timeout timeout = TimeoutSauce(connect=connect, read=read) except ValueError: raise ValueError( f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, " f"or a single float to set both timeouts to the same value." ) elif isinstance(timeout, TimeoutSauce): pass else: timeout = TimeoutSauce(connect=timeout, read=timeout) try: > resp = conn.urlopen( method=request.method, url=url, body=request.body, headers=request.headers, redirect=False, assert_same_host=False, preload_content=False, decode_content=False, retries=self.max_retries, timeout=timeout, chunked=chunked, ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:486: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff54084fbe0> method = 'POST', url = '/user/credentials/cdis/access_token' body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...NPva0i6TdvsONgWYV1N3mET3cX78Px5n6SH84sTMCLqWr5EOzw4ImhcQ2kOYfMOqKA", "key_id": "92c93027-24d6-4240-825c-30b9395d78cf"}' headers = {'User-Agent': 'python-requests/2.31.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'} retries = Retry(total=0, connect=None, read=False, redirect=None, status=None) redirect = False, assert_same_host = False timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None release_conn = False, chunked = False, body_pos = None response_kw = {'decode_content': False, 'preload_content': False} parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None) destination_scheme = None, conn = None, release_this_conn = True http_tunnel_required = False, err = None, clean_exit = False def urlopen( self, method, url, body=None, headers=None, retries=None, redirect=True, assert_same_host=True, timeout=_Default, pool_timeout=None, release_conn=None, chunked=False, body_pos=None, **response_kw ): """ Get a connection from the pool and perform an HTTP request. This is the lowest level call for making a request, so you'll need to specify all the raw details. .. note:: More commonly, it's appropriate to use a convenience method provided by :class:`.RequestMethods`, such as :meth:`request`. .. note:: `release_conn` will only behave as expected if `preload_content=False` because we want to make `preload_content=False` the default behaviour someday soon without breaking backwards compatibility. :param method: HTTP request method (such as GET, POST, PUT, etc.) :param url: The URL to perform the request on. :param body: Data to send in the request body, either :class:`str`, :class:`bytes`, an iterable of :class:`str`/:class:`bytes`, or a file-like object. :param headers: Dictionary of custom headers to send, such as User-Agent, If-None-Match, etc. If None, pool headers are used. If provided, these headers completely replace any pool-specific headers. :param retries: Configure the number of retries to allow before raising a :class:`~urllib3.exceptions.MaxRetryError` exception. Pass ``None`` to retry until you receive a response. Pass a :class:`~urllib3.util.retry.Retry` object for fine-grained control over different types of retries. Pass an integer number to retry connection errors that many times, but no other types of errors. Pass zero to never retry. If ``False``, then retries are disabled and any exception is raised immediately. Also, instead of raising a MaxRetryError on redirects, the redirect response will be returned. :type retries: :class:`~urllib3.util.retry.Retry`, False, or an int. :param redirect: If True, automatically handle redirects (status codes 301, 302, 303, 307, 308). Each redirect counts as a retry. Disabling retries will disable redirect, too. :param assert_same_host: If ``True``, will make sure that the host of the pool requests is consistent else will raise HostChangedError. When ``False``, you can use the pool on an HTTP proxy and request foreign hosts. :param timeout: If specified, overrides the default timeout for this one request. It may be a float (in seconds) or an instance of :class:`urllib3.util.Timeout`. :param pool_timeout: If set and the pool is set to block=True, then this method will block for ``pool_timeout`` seconds and raise EmptyPoolError if no connection is available within the time period. :param release_conn: If False, then the urlopen call will not release the connection back into the pool once a response is received (but will release if you read the entire contents of the response such as when `preload_content=True`). This is useful if you're not preloading the response's content immediately. You will need to call ``r.release_conn()`` on the response ``r`` to return the connection back into the pool. If None, it takes the value of ``response_kw.get('preload_content', True)``. :param chunked: If True, urllib3 will send the body using chunked transfer encoding. Otherwise, urllib3 will send the body using the standard content-length form. Defaults to False. :param int body_pos: Position to seek to in file-like body in the event of a retry or redirect. Typically this won't need to be set because urllib3 will auto-populate the value when needed. :param \\**response_kw: Additional parameters are passed to :meth:`urllib3.response.HTTPResponse.from_httplib` """ parsed_url = parse_url(url) destination_scheme = parsed_url.scheme if headers is None: headers = self.headers if not isinstance(retries, Retry): retries = Retry.from_int(retries, redirect=redirect, default=self.retries) if release_conn is None: release_conn = response_kw.get("preload_content", True) # Check host if assert_same_host and not self.is_same_host(url): raise HostChangedError(self, url, retries) # Ensure that the URL we're connecting to is properly encoded if url.startswith("/"): url = six.ensure_str(_encode_target(url)) else: url = six.ensure_str(parsed_url.url) conn = None # Track whether `conn` needs to be released before # returning/raising/recursing. Update this variable if necessary, and # leave `release_conn` constant throughout the function. That way, if # the function recurses, the original value of `release_conn` will be # passed down into the recursive call, and its value will be respected. # # See issue #651 [1] for details. # # [1] <https://github.com/urllib3/urllib3/issues/651> release_this_conn = release_conn http_tunnel_required = connection_requires_http_tunnel( self.proxy, self.proxy_config, destination_scheme ) # Merge the proxy headers. Only done when not using HTTP CONNECT. We # have to copy the headers dict so we can safely change it without those # changes being reflected in anyone else's copy. if not http_tunnel_required: headers = headers.copy() headers.update(self.proxy_headers) # Must keep the exception bound to a separate variable or else Python 3 # complains about UnboundLocalError. err = None # Keep track of whether we cleanly exited the except block. This # ensures we do proper cleanup in finally. clean_exit = False # Rewind body position, if needed. Record current position # for future rewinds in the event of a redirect/retry. body_pos = set_file_position(body, body_pos) try: # Request a connection from the queue. timeout_obj = self._get_timeout(timeout) conn = self._get_conn(timeout=pool_timeout) conn.timeout = timeout_obj.connect_timeout is_new_proxy_conn = self.proxy is not None and not getattr( conn, "sock", None ) if is_new_proxy_conn and http_tunnel_required: self._prepare_proxy(conn) # Make the request on the httplib connection object. httplib_response = self._make_request( conn, method, url, timeout=timeout_obj, body=body, headers=headers, chunked=chunked, ) # If we're going to release the connection in ``finally:``, then # the response doesn't need to know about the connection. Otherwise # it will also try to release it and we'll have a double-release # mess. response_conn = conn if not release_conn else None # Pass method to Response for length checking response_kw["request_method"] = method # Import httplib's response into our own wrapper object response = self.ResponseCls.from_httplib( httplib_response, pool=self, connection=response_conn, retries=retries, **response_kw ) # Everything went great! clean_exit = True except EmptyPoolError: # Didn't get a connection from the pool, no need to clean up clean_exit = True release_this_conn = False raise except ( TimeoutError, HTTPException, SocketError, ProtocolError, BaseSSLError, SSLError, CertificateError, ) as e: # Discard the connection for these exceptions. It will be # replaced during the next _get_conn() call. clean_exit = False def _is_ssl_error_message_from_http_proxy(ssl_error): # We're trying to detect the message 'WRONG_VERSION_NUMBER' but # SSLErrors are kinda all over the place when it comes to the message, # so we try to cover our bases here! message = " ".join(re.split("[^a-z]", str(ssl_error).lower())) return ( "wrong version number" in message or "unknown protocol" in message ) # Try to detect a common user error with proxies which is to # set an HTTP proxy to be HTTPS when it should be 'http://' # (ie {'http': 'http://proxy', 'https': 'https://proxy'}) # Instead we add a nice error message and point to a URL. if ( isinstance(e, BaseSSLError) and self.proxy and _is_ssl_error_message_from_http_proxy(e) and conn.proxy and conn.proxy.scheme == "https" ): e = ProxyError( "Your proxy appears to only use HTTP and not HTTPS, " "try changing your proxy URL to be HTTP. See: " "https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html" "#https-proxy-error-http-proxy", SSLError(e), ) elif isinstance(e, (BaseSSLError, CertificateError)): e = SSLError(e) elif isinstance(e, (SocketError, NewConnectionError)) and self.proxy: e = ProxyError("Cannot connect to proxy.", e) elif isinstance(e, (SocketError, HTTPException)): e = ProtocolError("Connection aborted.", e) > retries = retries.increment( method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:799: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = Retry(total=0, connect=None, read=False, redirect=None, status=None) method = 'POST', url = '/user/credentials/cdis/access_token', response = None error = SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133)')) _pool = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff54084fbe0> _stacktrace = <traceback object at 0x7ff541f6d400> def increment( self, method=None, url=None, response=None, error=None, _pool=None, _stacktrace=None, ): """Return a new Retry object with incremented retry counters. :param response: A response object, or None, if the server did not return a response. :type response: :class:`~urllib3.response.HTTPResponse` :param Exception error: An error encountered during the request, or None if the response was received successfully. :return: A new ``Retry`` object. """ if self.total is False and error: # Disabled, indicate to re-raise the error. raise six.reraise(type(error), error, _stacktrace) total = self.total if total is not None: total -= 1 connect = self.connect read = self.read redirect = self.redirect status_count = self.status other = self.other cause = "unknown" status = None redirect_location = None if error and self._is_connection_error(error): # Connect retry? if connect is False: raise six.reraise(type(error), error, _stacktrace) elif connect is not None: connect -= 1 elif error and self._is_read_error(error): # Read retry? if read is False or not self._is_method_retryable(method): raise six.reraise(type(error), error, _stacktrace) elif read is not None: read -= 1 elif error: # Other retry? if other is not None: other -= 1 elif response and response.get_redirect_location(): # Redirect retry? if redirect is not None: redirect -= 1 cause = "too many redirects" redirect_location = response.get_redirect_location() status = response.status else: # Incrementing because of a server error like a 500 in # status_forcelist and the given method is in the allowed_methods cause = ResponseError.GENERIC_ERROR if response and response.status: if status_count is not None: status_count -= 1 cause = ResponseError.SPECIFIC_ERROR.format(status_code=response.status) status = response.status history = self.history + ( RequestHistory(method, url, error, status, redirect_location), ) new_retry = self.new( total=total, connect=connect, read=read, redirect=redirect, status=status_count, other=other, history=history, ) if new_retry.is_exhausted(): > raise MaxRetryError(_pool, url, error or ResponseError(cause)) E urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133)'))) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/retry.py:592: MaxRetryError During handling of the above exception, another exception occurred: client = <starlette.testclient.TestClient object at 0x7ff540891b50> token = {'access_token': 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZGVudGl0eSI6InB1YmxpYyIsInNjb3BlIjpbImRlbW8xLTEyTCJdLCJleHBpcmUiOiJOb25lIn0.doEgWg8P2u6dGhajC1CBwJHS3TtbJh4L_vhMfK75uJQ'} def test_dataset_description_node(client, token): UUID = "5b9ae1bd-e780-4869-a458-b3422084c480" > response = client.get( f"/record/{UUID}", headers={"Authorization": f"Bearer {token['access_token']}"}, ) tests/test_data/test_gen3_data.py:72: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:602: in get return self.request("GET", url, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:476: in request return super().request( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:589: in request resp = self.send(prep, **send_kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:703: in send r = adapter.send(request, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:270: in send raise exc ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:267: in send portal.call(self.app, scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:283: in call return cast(T_Retval, self.start_task_soon(func, *args).result()) /usr/lib/python3.9/concurrent/futures/_base.py:446: in result return self.__get_result() /usr/lib/python3.9/concurrent/futures/_base.py:391: in __get_result raise self._exception ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:219: in _call_func retval = await retval ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/applications.py:269: in __call__ await super().__call__(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/applications.py:124: in __call__ await self.middleware_stack(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:184: in __call__ raise exc ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:162: in __call__ await self.app(scope, receive, _send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/cors.py:84: in __call__ await self.app(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:93: in __call__ raise exc ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:82: in __call__ await self.app(scope, receive, sender) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:21: in __call__ raise e ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:18: in __call__ await self.app(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:670: in __call__ await route.handle(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:266: in handle await self.app(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:65: in app response = await func(request) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:227: in app raw_response = await run_endpoint_function( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:160: in run_endpoint_function return await dependant.call(**values) app/main.py:318: in get_gen3_record record = connection["gen3"].export_record(program, project, uuid, "json") ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/submission.py:348: in export_record output = requests.get(api_url, auth=self._auth_provider).text ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:73: in get return request("get", url, params=params, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request return session.request(method=method, url=url, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:575: in request prep = self.prepare_request(req) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:486: in prepare_request p.prepare( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:372: in prepare self.prepare_auth(auth, url) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:603: in prepare_auth r = auth(self) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:325: in __call__ request.headers["Authorization"] = self._get_auth_value() ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:446: in _get_auth_value return "bearer " + self.get_access_token() ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:433: in get_access_token return self.refresh_access_token( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:367: in refresh_access_token self._access_token = get_access_token_with_key(self._refresh_token) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:75: in get_access_token_with_key resp = requests.post(auth_url, json=api_key) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:115: in post return request("post", url, data=data, json=json, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request return session.request(method=method, url=url, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:589: in request resp = self.send(prep, **send_kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:703: in send r = adapter.send(request, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <requests.adapters.HTTPAdapter object at 0x7ff54084f790> request = <PreparedRequest [POST]>, stream = False timeout = Timeout(connect=None, read=None, total=None), verify = True cert = None, proxies = OrderedDict() def send( self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None ): """Sends PreparedRequest object. Returns Response object. :param request: The :class:`PreparedRequest <PreparedRequest>` being sent. :param stream: (optional) Whether to stream the request content. :param timeout: (optional) How long to wait for the server to send data before giving up, as a float, or a :ref:`(connect timeout, read timeout) <timeouts>` tuple. :type timeout: float or tuple or urllib3 Timeout object :param verify: (optional) Either a boolean, in which case it controls whether we verify the server's TLS certificate, or a string, in which case it must be a path to a CA bundle to use :param cert: (optional) Any user-provided SSL certificate to be trusted. :param proxies: (optional) The proxies dictionary to apply to the request. :rtype: requests.Response """ try: conn = self.get_connection(request.url, proxies) except LocationValueError as e: raise InvalidURL(e, request=request) self.cert_verify(conn, request.url, verify, cert) url = self.request_url(request, proxies) self.add_headers( request, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies, ) chunked = not (request.body is None or "Content-Length" in request.headers) if isinstance(timeout, tuple): try: connect, read = timeout timeout = TimeoutSauce(connect=connect, read=read) except ValueError: raise ValueError( f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, " f"or a single float to set both timeouts to the same value." ) elif isinstance(timeout, TimeoutSauce): pass else: timeout = TimeoutSauce(connect=timeout, read=timeout) try: resp = conn.urlopen( method=request.method, url=url, body=request.body, headers=request.headers, redirect=False, assert_same_host=False, preload_content=False, decode_content=False, retries=self.max_retries, timeout=timeout, chunked=chunked, ) except (ProtocolError, OSError) as err: raise ConnectionError(err, request=request) except MaxRetryError as e: if isinstance(e.reason, ConnectTimeoutError): # TODO: Remove this in 3.0.0: see #2811 if not isinstance(e.reason, NewConnectionError): raise ConnectTimeout(e, request=request) if isinstance(e.reason, ResponseError): raise RetryError(e, request=request) if isinstance(e.reason, _ProxyError): raise ProxyError(e, request=request) if isinstance(e.reason, _SSLError): # This branch is for urllib3 v1.22 and later. > raise SSLError(e, request=request) E requests.exceptions.SSLError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133)'))) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:517: SSLError ------------------------------ Captured log setup ------------------------------ ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. INFO app.main:main.py:166 {'gen3': False, 'irods': True, 'orthanc': True} WARNING app.main:main.py:185 Failed to update default filter. ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. ------------------------------ Captured log call ------------------------------- ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. ______________________________ test_manifest_node ______________________________ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff540907ca0> method = 'POST', url = '/user/credentials/cdis/access_token' body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...NPva0i6TdvsONgWYV1N3mET3cX78Px5n6SH84sTMCLqWr5EOzw4ImhcQ2kOYfMOqKA", "key_id": "92c93027-24d6-4240-825c-30b9395d78cf"}' headers = {'User-Agent': 'python-requests/2.31.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'} retries = Retry(total=0, connect=None, read=False, redirect=None, status=None) redirect = False, assert_same_host = False timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None release_conn = False, chunked = False, body_pos = None response_kw = {'decode_content': False, 'preload_content': False} parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None) destination_scheme = None, conn = None, release_this_conn = True http_tunnel_required = False, err = None, clean_exit = False def urlopen( self, method, url, body=None, headers=None, retries=None, redirect=True, assert_same_host=True, timeout=_Default, pool_timeout=None, release_conn=None, chunked=False, body_pos=None, **response_kw ): """ Get a connection from the pool and perform an HTTP request. This is the lowest level call for making a request, so you'll need to specify all the raw details. .. note:: More commonly, it's appropriate to use a convenience method provided by :class:`.RequestMethods`, such as :meth:`request`. .. note:: `release_conn` will only behave as expected if `preload_content=False` because we want to make `preload_content=False` the default behaviour someday soon without breaking backwards compatibility. :param method: HTTP request method (such as GET, POST, PUT, etc.) :param url: The URL to perform the request on. :param body: Data to send in the request body, either :class:`str`, :class:`bytes`, an iterable of :class:`str`/:class:`bytes`, or a file-like object. :param headers: Dictionary of custom headers to send, such as User-Agent, If-None-Match, etc. If None, pool headers are used. If provided, these headers completely replace any pool-specific headers. :param retries: Configure the number of retries to allow before raising a :class:`~urllib3.exceptions.MaxRetryError` exception. Pass ``None`` to retry until you receive a response. Pass a :class:`~urllib3.util.retry.Retry` object for fine-grained control over different types of retries. Pass an integer number to retry connection errors that many times, but no other types of errors. Pass zero to never retry. If ``False``, then retries are disabled and any exception is raised immediately. Also, instead of raising a MaxRetryError on redirects, the redirect response will be returned. :type retries: :class:`~urllib3.util.retry.Retry`, False, or an int. :param redirect: If True, automatically handle redirects (status codes 301, 302, 303, 307, 308). Each redirect counts as a retry. Disabling retries will disable redirect, too. :param assert_same_host: If ``True``, will make sure that the host of the pool requests is consistent else will raise HostChangedError. When ``False``, you can use the pool on an HTTP proxy and request foreign hosts. :param timeout: If specified, overrides the default timeout for this one request. It may be a float (in seconds) or an instance of :class:`urllib3.util.Timeout`. :param pool_timeout: If set and the pool is set to block=True, then this method will block for ``pool_timeout`` seconds and raise EmptyPoolError if no connection is available within the time period. :param release_conn: If False, then the urlopen call will not release the connection back into the pool once a response is received (but will release if you read the entire contents of the response such as when `preload_content=True`). This is useful if you're not preloading the response's content immediately. You will need to call ``r.release_conn()`` on the response ``r`` to return the connection back into the pool. If None, it takes the value of ``response_kw.get('preload_content', True)``. :param chunked: If True, urllib3 will send the body using chunked transfer encoding. Otherwise, urllib3 will send the body using the standard content-length form. Defaults to False. :param int body_pos: Position to seek to in file-like body in the event of a retry or redirect. Typically this won't need to be set because urllib3 will auto-populate the value when needed. :param \\**response_kw: Additional parameters are passed to :meth:`urllib3.response.HTTPResponse.from_httplib` """ parsed_url = parse_url(url) destination_scheme = parsed_url.scheme if headers is None: headers = self.headers if not isinstance(retries, Retry): retries = Retry.from_int(retries, redirect=redirect, default=self.retries) if release_conn is None: release_conn = response_kw.get("preload_content", True) # Check host if assert_same_host and not self.is_same_host(url): raise HostChangedError(self, url, retries) # Ensure that the URL we're connecting to is properly encoded if url.startswith("/"): url = six.ensure_str(_encode_target(url)) else: url = six.ensure_str(parsed_url.url) conn = None # Track whether `conn` needs to be released before # returning/raising/recursing. Update this variable if necessary, and # leave `release_conn` constant throughout the function. That way, if # the function recurses, the original value of `release_conn` will be # passed down into the recursive call, and its value will be respected. # # See issue #651 [1] for details. # # [1] <https://github.com/urllib3/urllib3/issues/651> release_this_conn = release_conn http_tunnel_required = connection_requires_http_tunnel( self.proxy, self.proxy_config, destination_scheme ) # Merge the proxy headers. Only done when not using HTTP CONNECT. We # have to copy the headers dict so we can safely change it without those # changes being reflected in anyone else's copy. if not http_tunnel_required: headers = headers.copy() headers.update(self.proxy_headers) # Must keep the exception bound to a separate variable or else Python 3 # complains about UnboundLocalError. err = None # Keep track of whether we cleanly exited the except block. This # ensures we do proper cleanup in finally. clean_exit = False # Rewind body position, if needed. Record current position # for future rewinds in the event of a redirect/retry. body_pos = set_file_position(body, body_pos) try: # Request a connection from the queue. timeout_obj = self._get_timeout(timeout) conn = self._get_conn(timeout=pool_timeout) conn.timeout = timeout_obj.connect_timeout is_new_proxy_conn = self.proxy is not None and not getattr( conn, "sock", None ) if is_new_proxy_conn and http_tunnel_required: self._prepare_proxy(conn) # Make the request on the httplib connection object. > httplib_response = self._make_request( conn, method, url, timeout=timeout_obj, body=body, headers=headers, chunked=chunked, ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:715: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff540907ca0> conn = <urllib3.connection.HTTPSConnection object at 0x7ff540907100> method = 'POST', url = '/user/credentials/cdis/access_token' timeout = Timeout(connect=None, read=None, total=None), chunked = False httplib_request_kw = {'body': b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwd...p, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}} timeout_obj = Timeout(connect=None, read=None, total=None) def _make_request( self, conn, method, url, timeout=_Default, chunked=False, **httplib_request_kw ): """ Perform a request on a given urllib connection object taken from our pool. :param conn: a connection from one of our connection pools :param timeout: Socket timeout in seconds for the request. This can be a float or integer, which will set the same timeout value for the socket connect and the socket read, or an instance of :class:`urllib3.util.Timeout`, which gives you more fine-grained control over your timeouts. """ self.num_requests += 1 timeout_obj = self._get_timeout(timeout) timeout_obj.start_connect() conn.timeout = Timeout.resolve_default_timeout(timeout_obj.connect_timeout) # Trigger any extra validation we need to do. try: > self._validate_conn(conn) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:404: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff540907ca0> conn = <urllib3.connection.HTTPSConnection object at 0x7ff540907100> def _validate_conn(self, conn): """ Called right before a request is made, after the socket is created. """ super(HTTPSConnectionPool, self)._validate_conn(conn) # Force connect early to allow us to validate the connection. if not getattr(conn, "sock", None): # AppEngine might not have `.sock` > conn.connect() ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:1058: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <urllib3.connection.HTTPSConnection object at 0x7ff540907100> def connect(self): # Add certificate verification self.sock = conn = self._new_conn() hostname = self.host tls_in_tls = False if self._is_using_tunnel(): if self.tls_in_tls_required: self.sock = conn = self._connect_tls_proxy(hostname, conn) tls_in_tls = True # Calls self._set_hostport(), so self.host is # self._tunnel_host below. self._tunnel() # Mark this connection as not reusable self.auto_open = 0 # Override the host with the one we're requesting data from. hostname = self._tunnel_host server_hostname = hostname if self.server_hostname is not None: server_hostname = self.server_hostname is_time_off = datetime.date.today() < RECENT_DATE if is_time_off: warnings.warn( ( "System time is way off (before {0}). This will probably " "lead to SSL verification errors" ).format(RECENT_DATE), SystemTimeWarning, ) # Wrap socket using verification with the root certs in # trusted_root_certs default_ssl_context = False if self.ssl_context is None: default_ssl_context = True self.ssl_context = create_urllib3_context( ssl_version=resolve_ssl_version(self.ssl_version), cert_reqs=resolve_cert_reqs(self.cert_reqs), ) context = self.ssl_context context.verify_mode = resolve_cert_reqs(self.cert_reqs) # Try to load OS default certs if none are given. # Works well on Windows (requires Python3.4+) if ( not self.ca_certs and not self.ca_cert_dir and not self.ca_cert_data and default_ssl_context and hasattr(context, "load_default_certs") ): context.load_default_certs() > self.sock = ssl_wrap_socket( sock=conn, keyfile=self.key_file, certfile=self.cert_file, key_password=self.key_password, ca_certs=self.ca_certs, ca_cert_dir=self.ca_cert_dir, ca_cert_data=self.ca_cert_data, server_hostname=server_hostname, ssl_context=context, tls_in_tls=tls_in_tls, ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:419: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> keyfile = None, certfile = None, cert_reqs = None ca_certs = '/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/certifi/cacert.pem' server_hostname = 'gen3.abi-ctt-ctp.cloud.edu.au', ssl_version = None ciphers = None, ssl_context = <ssl.SSLContext object at 0x7ff5409c4640> ca_cert_dir = None, key_password = None, ca_cert_data = None, tls_in_tls = False def ssl_wrap_socket( sock, keyfile=None, certfile=None, cert_reqs=None, ca_certs=None, server_hostname=None, ssl_version=None, ciphers=None, ssl_context=None, ca_cert_dir=None, key_password=None, ca_cert_data=None, tls_in_tls=False, ): """ All arguments except for server_hostname, ssl_context, and ca_cert_dir have the same meaning as they do when using :func:`ssl.wrap_socket`. :param server_hostname: When SNI is supported, the expected hostname of the certificate :param ssl_context: A pre-made :class:`SSLContext` object. If none is provided, one will be created using :func:`create_urllib3_context`. :param ciphers: A string of ciphers we wish the client to support. :param ca_cert_dir: A directory containing CA certificates in multiple separate files, as supported by OpenSSL's -CApath flag or the capath argument to SSLContext.load_verify_locations(). :param key_password: Optional password if the keyfile is encrypted. :param ca_cert_data: Optional string containing CA certificates in PEM format suitable for passing as the cadata parameter to SSLContext.load_verify_locations() :param tls_in_tls: Use SSLTransport to wrap the existing socket. """ context = ssl_context if context is None: # Note: This branch of code and all the variables in it are no longer # used by urllib3 itself. We should consider deprecating and removing # this code. context = create_urllib3_context(ssl_version, cert_reqs, ciphers=ciphers) if ca_certs or ca_cert_dir or ca_cert_data: try: context.load_verify_locations(ca_certs, ca_cert_dir, ca_cert_data) except (IOError, OSError) as e: raise SSLError(e) elif ssl_context is None and hasattr(context, "load_default_certs"): # try to load OS default certs; works well on Windows (require Python3.4+) context.load_default_certs() # Attempt to detect if we get the goofy behavior of the # keyfile being encrypted and OpenSSL asking for the # passphrase via the terminal and instead error out. if keyfile and key_password is None and _is_key_file_encrypted(keyfile): raise SSLError("Client private key is encrypted, password is required") if certfile: if key_password is None: context.load_cert_chain(certfile, keyfile) else: context.load_cert_chain(certfile, keyfile, key_password) try: if hasattr(context, "set_alpn_protocols"): context.set_alpn_protocols(ALPN_PROTOCOLS) except NotImplementedError: # Defensive: in CI, we always have set_alpn_protocols pass # If we detect server_hostname is an IP address then the SNI # extension should not be used according to RFC3546 Section 3.1 use_sni_hostname = server_hostname and not is_ipaddress(server_hostname) # SecureTransport uses server_hostname in certificate verification. send_sni = (use_sni_hostname and HAS_SNI) or ( IS_SECURETRANSPORT and server_hostname ) # Do not warn the user if server_hostname is an invalid SNI hostname. if not HAS_SNI and use_sni_hostname: warnings.warn( "An HTTPS request has been made, but the SNI (Server Name " "Indication) extension to TLS is not available on this platform. " "This may cause the server to present an incorrect TLS " "certificate, which can cause validation failures. You can upgrade to " "a newer version of Python to solve this. For more information, see " "https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html" "#ssl-warnings", SNIMissingWarning, ) if send_sni: > ssl_sock = _ssl_wrap_socket_impl( sock, context, tls_in_tls, server_hostname=server_hostname ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/ssl_.py:449: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> ssl_context = <ssl.SSLContext object at 0x7ff5409c4640>, tls_in_tls = False server_hostname = 'gen3.abi-ctt-ctp.cloud.edu.au' def _ssl_wrap_socket_impl(sock, ssl_context, tls_in_tls, server_hostname=None): if tls_in_tls: if not SSLTransport: # Import error, ssl is not available. raise ProxySchemeUnsupported( "TLS in TLS requires support for the 'ssl' module" ) SSLTransport._validate_ssl_context_for_tls_in_tls(ssl_context) return SSLTransport(sock, ssl_context, server_hostname) if server_hostname: > return ssl_context.wrap_socket(sock, server_hostname=server_hostname) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/ssl_.py:493: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <ssl.SSLContext object at 0x7ff5409c4640> sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> server_side = False, do_handshake_on_connect = True, suppress_ragged_eofs = True server_hostname = 'gen3.abi-ctt-ctp.cloud.edu.au', session = None def wrap_socket(self, sock, server_side=False, do_handshake_on_connect=True, suppress_ragged_eofs=True, server_hostname=None, session=None): # SSLSocket class handles server_hostname encoding before it calls # ctx._wrap_socket() > return self.sslsocket_class._create( sock=sock, server_side=server_side, do_handshake_on_connect=do_handshake_on_connect, suppress_ragged_eofs=suppress_ragged_eofs, server_hostname=server_hostname, context=self, session=session ) /usr/lib/python3.9/ssl.py:501: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ cls = <class 'ssl.SSLSocket'> sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> server_side = False, do_handshake_on_connect = True, suppress_ragged_eofs = True server_hostname = 'gen3.abi-ctt-ctp.cloud.edu.au' context = <ssl.SSLContext object at 0x7ff5409c4640>, session = None @classmethod def _create(cls, sock, server_side=False, do_handshake_on_connect=True, suppress_ragged_eofs=True, server_hostname=None, context=None, session=None): if sock.getsockopt(SOL_SOCKET, SO_TYPE) != SOCK_STREAM: raise NotImplementedError("only stream sockets are supported") if server_side: if server_hostname: raise ValueError("server_hostname can only be specified " "in client mode") if session is not None: raise ValueError("session can only be specified in " "client mode") if context.check_hostname and not server_hostname: raise ValueError("check_hostname requires server_hostname") kwargs = dict( family=sock.family, type=sock.type, proto=sock.proto, fileno=sock.fileno() ) self = cls.__new__(cls, **kwargs) super(SSLSocket, self).__init__(**kwargs) sock_timeout = sock.gettimeout() sock.detach() self._context = context self._session = session self._closed = False self._sslobj = None self.server_side = server_side self.server_hostname = context._encode_hostname(server_hostname) self.do_handshake_on_connect = do_handshake_on_connect self.suppress_ragged_eofs = suppress_ragged_eofs # See if we are connected try: self.getpeername() except OSError as e: if e.errno != errno.ENOTCONN: raise connected = False blocking = self.getblocking() self.setblocking(False) try: # We are not connected so this is not supposed to block, but # testing revealed otherwise on macOS and Windows so we do # the non-blocking dance regardless. Our raise when any data # is found means consuming the data is harmless. notconn_pre_handshake_data = self.recv(1) except OSError as e: # EINVAL occurs for recv(1) on non-connected on unix sockets. if e.errno not in (errno.ENOTCONN, errno.EINVAL): raise notconn_pre_handshake_data = b'' self.setblocking(blocking) if notconn_pre_handshake_data: # This prevents pending data sent to the socket before it was # closed from escaping to the caller who could otherwise # presume it came through a successful TLS connection. reason = "Closed before TLS handshake with data in recv buffer." notconn_pre_handshake_data_error = SSLError(e.errno, reason) # Add the SSLError attributes that _ssl.c always adds. notconn_pre_handshake_data_error.reason = reason notconn_pre_handshake_data_error.library = None try: self.close() except OSError: pass try: raise notconn_pre_handshake_data_error finally: # Explicitly break the reference cycle. notconn_pre_handshake_data_error = None else: connected = True self.settimeout(sock_timeout) # Must come after setblocking() calls. self._connected = connected if connected: # create the SSL object try: self._sslobj = self._context._wrap_socket( self, server_side, self.server_hostname, owner=self, session=self._session, ) if do_handshake_on_connect: timeout = self.gettimeout() if timeout == 0.0: # non-blocking raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets") > self.do_handshake() /usr/lib/python3.9/ssl.py:1074: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <ssl.SSLSocket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> block = False @_sslcopydoc def do_handshake(self, block=False): self._check_connected() timeout = self.gettimeout() try: if timeout == 0.0 and block: self.settimeout(None) > self._sslobj.do_handshake() E ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133) /usr/lib/python3.9/ssl.py:1343: SSLCertVerificationError During handling of the above exception, another exception occurred: self = <requests.adapters.HTTPAdapter object at 0x7ff540da2940> request = <PreparedRequest [POST]>, stream = False timeout = Timeout(connect=None, read=None, total=None), verify = True cert = None, proxies = OrderedDict() def send( self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None ): """Sends PreparedRequest object. Returns Response object. :param request: The :class:`PreparedRequest <PreparedRequest>` being sent. :param stream: (optional) Whether to stream the request content. :param timeout: (optional) How long to wait for the server to send data before giving up, as a float, or a :ref:`(connect timeout, read timeout) <timeouts>` tuple. :type timeout: float or tuple or urllib3 Timeout object :param verify: (optional) Either a boolean, in which case it controls whether we verify the server's TLS certificate, or a string, in which case it must be a path to a CA bundle to use :param cert: (optional) Any user-provided SSL certificate to be trusted. :param proxies: (optional) The proxies dictionary to apply to the request. :rtype: requests.Response """ try: conn = self.get_connection(request.url, proxies) except LocationValueError as e: raise InvalidURL(e, request=request) self.cert_verify(conn, request.url, verify, cert) url = self.request_url(request, proxies) self.add_headers( request, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies, ) chunked = not (request.body is None or "Content-Length" in request.headers) if isinstance(timeout, tuple): try: connect, read = timeout timeout = TimeoutSauce(connect=connect, read=read) except ValueError: raise ValueError( f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, " f"or a single float to set both timeouts to the same value." ) elif isinstance(timeout, TimeoutSauce): pass else: timeout = TimeoutSauce(connect=timeout, read=timeout) try: > resp = conn.urlopen( method=request.method, url=url, body=request.body, headers=request.headers, redirect=False, assert_same_host=False, preload_content=False, decode_content=False, retries=self.max_retries, timeout=timeout, chunked=chunked, ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:486: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff540907ca0> method = 'POST', url = '/user/credentials/cdis/access_token' body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...NPva0i6TdvsONgWYV1N3mET3cX78Px5n6SH84sTMCLqWr5EOzw4ImhcQ2kOYfMOqKA", "key_id": "92c93027-24d6-4240-825c-30b9395d78cf"}' headers = {'User-Agent': 'python-requests/2.31.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'} retries = Retry(total=0, connect=None, read=False, redirect=None, status=None) redirect = False, assert_same_host = False timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None release_conn = False, chunked = False, body_pos = None response_kw = {'decode_content': False, 'preload_content': False} parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None) destination_scheme = None, conn = None, release_this_conn = True http_tunnel_required = False, err = None, clean_exit = False def urlopen( self, method, url, body=None, headers=None, retries=None, redirect=True, assert_same_host=True, timeout=_Default, pool_timeout=None, release_conn=None, chunked=False, body_pos=None, **response_kw ): """ Get a connection from the pool and perform an HTTP request. This is the lowest level call for making a request, so you'll need to specify all the raw details. .. note:: More commonly, it's appropriate to use a convenience method provided by :class:`.RequestMethods`, such as :meth:`request`. .. note:: `release_conn` will only behave as expected if `preload_content=False` because we want to make `preload_content=False` the default behaviour someday soon without breaking backwards compatibility. :param method: HTTP request method (such as GET, POST, PUT, etc.) :param url: The URL to perform the request on. :param body: Data to send in the request body, either :class:`str`, :class:`bytes`, an iterable of :class:`str`/:class:`bytes`, or a file-like object. :param headers: Dictionary of custom headers to send, such as User-Agent, If-None-Match, etc. If None, pool headers are used. If provided, these headers completely replace any pool-specific headers. :param retries: Configure the number of retries to allow before raising a :class:`~urllib3.exceptions.MaxRetryError` exception. Pass ``None`` to retry until you receive a response. Pass a :class:`~urllib3.util.retry.Retry` object for fine-grained control over different types of retries. Pass an integer number to retry connection errors that many times, but no other types of errors. Pass zero to never retry. If ``False``, then retries are disabled and any exception is raised immediately. Also, instead of raising a MaxRetryError on redirects, the redirect response will be returned. :type retries: :class:`~urllib3.util.retry.Retry`, False, or an int. :param redirect: If True, automatically handle redirects (status codes 301, 302, 303, 307, 308). Each redirect counts as a retry. Disabling retries will disable redirect, too. :param assert_same_host: If ``True``, will make sure that the host of the pool requests is consistent else will raise HostChangedError. When ``False``, you can use the pool on an HTTP proxy and request foreign hosts. :param timeout: If specified, overrides the default timeout for this one request. It may be a float (in seconds) or an instance of :class:`urllib3.util.Timeout`. :param pool_timeout: If set and the pool is set to block=True, then this method will block for ``pool_timeout`` seconds and raise EmptyPoolError if no connection is available within the time period. :param release_conn: If False, then the urlopen call will not release the connection back into the pool once a response is received (but will release if you read the entire contents of the response such as when `preload_content=True`). This is useful if you're not preloading the response's content immediately. You will need to call ``r.release_conn()`` on the response ``r`` to return the connection back into the pool. If None, it takes the value of ``response_kw.get('preload_content', True)``. :param chunked: If True, urllib3 will send the body using chunked transfer encoding. Otherwise, urllib3 will send the body using the standard content-length form. Defaults to False. :param int body_pos: Position to seek to in file-like body in the event of a retry or redirect. Typically this won't need to be set because urllib3 will auto-populate the value when needed. :param \\**response_kw: Additional parameters are passed to :meth:`urllib3.response.HTTPResponse.from_httplib` """ parsed_url = parse_url(url) destination_scheme = parsed_url.scheme if headers is None: headers = self.headers if not isinstance(retries, Retry): retries = Retry.from_int(retries, redirect=redirect, default=self.retries) if release_conn is None: release_conn = response_kw.get("preload_content", True) # Check host if assert_same_host and not self.is_same_host(url): raise HostChangedError(self, url, retries) # Ensure that the URL we're connecting to is properly encoded if url.startswith("/"): url = six.ensure_str(_encode_target(url)) else: url = six.ensure_str(parsed_url.url) conn = None # Track whether `conn` needs to be released before # returning/raising/recursing. Update this variable if necessary, and # leave `release_conn` constant throughout the function. That way, if # the function recurses, the original value of `release_conn` will be # passed down into the recursive call, and its value will be respected. # # See issue #651 [1] for details. # # [1] <https://github.com/urllib3/urllib3/issues/651> release_this_conn = release_conn http_tunnel_required = connection_requires_http_tunnel( self.proxy, self.proxy_config, destination_scheme ) # Merge the proxy headers. Only done when not using HTTP CONNECT. We # have to copy the headers dict so we can safely change it without those # changes being reflected in anyone else's copy. if not http_tunnel_required: headers = headers.copy() headers.update(self.proxy_headers) # Must keep the exception bound to a separate variable or else Python 3 # complains about UnboundLocalError. err = None # Keep track of whether we cleanly exited the except block. This # ensures we do proper cleanup in finally. clean_exit = False # Rewind body position, if needed. Record current position # for future rewinds in the event of a redirect/retry. body_pos = set_file_position(body, body_pos) try: # Request a connection from the queue. timeout_obj = self._get_timeout(timeout) conn = self._get_conn(timeout=pool_timeout) conn.timeout = timeout_obj.connect_timeout is_new_proxy_conn = self.proxy is not None and not getattr( conn, "sock", None ) if is_new_proxy_conn and http_tunnel_required: self._prepare_proxy(conn) # Make the request on the httplib connection object. httplib_response = self._make_request( conn, method, url, timeout=timeout_obj, body=body, headers=headers, chunked=chunked, ) # If we're going to release the connection in ``finally:``, then # the response doesn't need to know about the connection. Otherwise # it will also try to release it and we'll have a double-release # mess. response_conn = conn if not release_conn else None # Pass method to Response for length checking response_kw["request_method"] = method # Import httplib's response into our own wrapper object response = self.ResponseCls.from_httplib( httplib_response, pool=self, connection=response_conn, retries=retries, **response_kw ) # Everything went great! clean_exit = True except EmptyPoolError: # Didn't get a connection from the pool, no need to clean up clean_exit = True release_this_conn = False raise except ( TimeoutError, HTTPException, SocketError, ProtocolError, BaseSSLError, SSLError, CertificateError, ) as e: # Discard the connection for these exceptions. It will be # replaced during the next _get_conn() call. clean_exit = False def _is_ssl_error_message_from_http_proxy(ssl_error): # We're trying to detect the message 'WRONG_VERSION_NUMBER' but # SSLErrors are kinda all over the place when it comes to the message, # so we try to cover our bases here! message = " ".join(re.split("[^a-z]", str(ssl_error).lower())) return ( "wrong version number" in message or "unknown protocol" in message ) # Try to detect a common user error with proxies which is to # set an HTTP proxy to be HTTPS when it should be 'http://' # (ie {'http': 'http://proxy', 'https': 'https://proxy'}) # Instead we add a nice error message and point to a URL. if ( isinstance(e, BaseSSLError) and self.proxy and _is_ssl_error_message_from_http_proxy(e) and conn.proxy and conn.proxy.scheme == "https" ): e = ProxyError( "Your proxy appears to only use HTTP and not HTTPS, " "try changing your proxy URL to be HTTP. See: " "https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html" "#https-proxy-error-http-proxy", SSLError(e), ) elif isinstance(e, (BaseSSLError, CertificateError)): e = SSLError(e) elif isinstance(e, (SocketError, NewConnectionError)) and self.proxy: e = ProxyError("Cannot connect to proxy.", e) elif isinstance(e, (SocketError, HTTPException)): e = ProtocolError("Connection aborted.", e) > retries = retries.increment( method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:799: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = Retry(total=0, connect=None, read=False, redirect=None, status=None) method = 'POST', url = '/user/credentials/cdis/access_token', response = None error = SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133)')) _pool = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff540907ca0> _stacktrace = <traceback object at 0x7ff5404cc980> def increment( self, method=None, url=None, response=None, error=None, _pool=None, _stacktrace=None, ): """Return a new Retry object with incremented retry counters. :param response: A response object, or None, if the server did not return a response. :type response: :class:`~urllib3.response.HTTPResponse` :param Exception error: An error encountered during the request, or None if the response was received successfully. :return: A new ``Retry`` object. """ if self.total is False and error: # Disabled, indicate to re-raise the error. raise six.reraise(type(error), error, _stacktrace) total = self.total if total is not None: total -= 1 connect = self.connect read = self.read redirect = self.redirect status_count = self.status other = self.other cause = "unknown" status = None redirect_location = None if error and self._is_connection_error(error): # Connect retry? if connect is False: raise six.reraise(type(error), error, _stacktrace) elif connect is not None: connect -= 1 elif error and self._is_read_error(error): # Read retry? if read is False or not self._is_method_retryable(method): raise six.reraise(type(error), error, _stacktrace) elif read is not None: read -= 1 elif error: # Other retry? if other is not None: other -= 1 elif response and response.get_redirect_location(): # Redirect retry? if redirect is not None: redirect -= 1 cause = "too many redirects" redirect_location = response.get_redirect_location() status = response.status else: # Incrementing because of a server error like a 500 in # status_forcelist and the given method is in the allowed_methods cause = ResponseError.GENERIC_ERROR if response and response.status: if status_count is not None: status_count -= 1 cause = ResponseError.SPECIFIC_ERROR.format(status_code=response.status) status = response.status history = self.history + ( RequestHistory(method, url, error, status, redirect_location), ) new_retry = self.new( total=total, connect=connect, read=read, redirect=redirect, status=status_count, other=other, history=history, ) if new_retry.is_exhausted(): > raise MaxRetryError(_pool, url, error or ResponseError(cause)) E urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133)'))) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/retry.py:592: MaxRetryError During handling of the above exception, another exception occurred: client = <starlette.testclient.TestClient object at 0x7ff540d9fa90> token = {'access_token': 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZGVudGl0eSI6InB1YmxpYyIsInNjb3BlIjpbImRlbW8xLTEyTCJdLCJleHBpcmUiOiJOb25lIn0.doEgWg8P2u6dGhajC1CBwJHS3TtbJh4L_vhMfK75uJQ'} def test_manifest_node(client, token): UUID = "fd65a93f-ff62-45e4-b7b6-96419ef4f749" > response = client.get( f"/record/{UUID}", headers={"Authorization": f"Bearer {token['access_token']}"}, ) tests/test_data/test_gen3_data.py:173: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:602: in get return self.request("GET", url, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:476: in request return super().request( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:589: in request resp = self.send(prep, **send_kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:703: in send r = adapter.send(request, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:270: in send raise exc ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:267: in send portal.call(self.app, scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:283: in call return cast(T_Retval, self.start_task_soon(func, *args).result()) /usr/lib/python3.9/concurrent/futures/_base.py:446: in result return self.__get_result() /usr/lib/python3.9/concurrent/futures/_base.py:391: in __get_result raise self._exception ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:219: in _call_func retval = await retval ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/applications.py:269: in __call__ await super().__call__(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/applications.py:124: in __call__ await self.middleware_stack(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:184: in __call__ raise exc ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:162: in __call__ await self.app(scope, receive, _send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/cors.py:84: in __call__ await self.app(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:93: in __call__ raise exc ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:82: in __call__ await self.app(scope, receive, sender) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:21: in __call__ raise e ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:18: in __call__ await self.app(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:670: in __call__ await route.handle(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:266: in handle await self.app(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:65: in app response = await func(request) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:227: in app raw_response = await run_endpoint_function( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:160: in run_endpoint_function return await dependant.call(**values) app/main.py:318: in get_gen3_record record = connection["gen3"].export_record(program, project, uuid, "json") ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/submission.py:348: in export_record output = requests.get(api_url, auth=self._auth_provider).text ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:73: in get return request("get", url, params=params, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request return session.request(method=method, url=url, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:575: in request prep = self.prepare_request(req) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:486: in prepare_request p.prepare( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:372: in prepare self.prepare_auth(auth, url) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:603: in prepare_auth r = auth(self) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:325: in __call__ request.headers["Authorization"] = self._get_auth_value() ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:446: in _get_auth_value return "bearer " + self.get_access_token() ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:433: in get_access_token return self.refresh_access_token( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:367: in refresh_access_token self._access_token = get_access_token_with_key(self._refresh_token) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:75: in get_access_token_with_key resp = requests.post(auth_url, json=api_key) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:115: in post return request("post", url, data=data, json=json, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request return session.request(method=method, url=url, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:589: in request resp = self.send(prep, **send_kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:703: in send r = adapter.send(request, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <requests.adapters.HTTPAdapter object at 0x7ff540da2940> request = <PreparedRequest [POST]>, stream = False timeout = Timeout(connect=None, read=None, total=None), verify = True cert = None, proxies = OrderedDict() def send( self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None ): """Sends PreparedRequest object. Returns Response object. :param request: The :class:`PreparedRequest <PreparedRequest>` being sent. :param stream: (optional) Whether to stream the request content. :param timeout: (optional) How long to wait for the server to send data before giving up, as a float, or a :ref:`(connect timeout, read timeout) <timeouts>` tuple. :type timeout: float or tuple or urllib3 Timeout object :param verify: (optional) Either a boolean, in which case it controls whether we verify the server's TLS certificate, or a string, in which case it must be a path to a CA bundle to use :param cert: (optional) Any user-provided SSL certificate to be trusted. :param proxies: (optional) The proxies dictionary to apply to the request. :rtype: requests.Response """ try: conn = self.get_connection(request.url, proxies) except LocationValueError as e: raise InvalidURL(e, request=request) self.cert_verify(conn, request.url, verify, cert) url = self.request_url(request, proxies) self.add_headers( request, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies, ) chunked = not (request.body is None or "Content-Length" in request.headers) if isinstance(timeout, tuple): try: connect, read = timeout timeout = TimeoutSauce(connect=connect, read=read) except ValueError: raise ValueError( f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, " f"or a single float to set both timeouts to the same value." ) elif isinstance(timeout, TimeoutSauce): pass else: timeout = TimeoutSauce(connect=timeout, read=timeout) try: resp = conn.urlopen( method=request.method, url=url, body=request.body, headers=request.headers, redirect=False, assert_same_host=False, preload_content=False, decode_content=False, retries=self.max_retries, timeout=timeout, chunked=chunked, ) except (ProtocolError, OSError) as err: raise ConnectionError(err, request=request) except MaxRetryError as e: if isinstance(e.reason, ConnectTimeoutError): # TODO: Remove this in 3.0.0: see #2811 if not isinstance(e.reason, NewConnectionError): raise ConnectTimeout(e, request=request) if isinstance(e.reason, ResponseError): raise RetryError(e, request=request) if isinstance(e.reason, _ProxyError): raise ProxyError(e, request=request) if isinstance(e.reason, _SSLError): # This branch is for urllib3 v1.22 and later. > raise SSLError(e, request=request) E requests.exceptions.SSLError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133)'))) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:517: SSLError ------------------------------ Captured log setup ------------------------------ ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. INFO app.main:main.py:166 {'gen3': False, 'irods': True, 'orthanc': True} WARNING app.main:main.py:185 Failed to update default filter. ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. ------------------------------ Captured log call ------------------------------- ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. ________________________________ test_case_node ________________________________ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff542148100> method = 'POST', url = '/user/credentials/cdis/access_token' body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...NPva0i6TdvsONgWYV1N3mET3cX78Px5n6SH84sTMCLqWr5EOzw4ImhcQ2kOYfMOqKA", "key_id": "92c93027-24d6-4240-825c-30b9395d78cf"}' headers = {'User-Agent': 'python-requests/2.31.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'} retries = Retry(total=0, connect=None, read=False, redirect=None, status=None) redirect = False, assert_same_host = False timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None release_conn = False, chunked = False, body_pos = None response_kw = {'decode_content': False, 'preload_content': False} parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None) destination_scheme = None, conn = None, release_this_conn = True http_tunnel_required = False, err = None, clean_exit = False def urlopen( self, method, url, body=None, headers=None, retries=None, redirect=True, assert_same_host=True, timeout=_Default, pool_timeout=None, release_conn=None, chunked=False, body_pos=None, **response_kw ): """ Get a connection from the pool and perform an HTTP request. This is the lowest level call for making a request, so you'll need to specify all the raw details. .. note:: More commonly, it's appropriate to use a convenience method provided by :class:`.RequestMethods`, such as :meth:`request`. .. note:: `release_conn` will only behave as expected if `preload_content=False` because we want to make `preload_content=False` the default behaviour someday soon without breaking backwards compatibility. :param method: HTTP request method (such as GET, POST, PUT, etc.) :param url: The URL to perform the request on. :param body: Data to send in the request body, either :class:`str`, :class:`bytes`, an iterable of :class:`str`/:class:`bytes`, or a file-like object. :param headers: Dictionary of custom headers to send, such as User-Agent, If-None-Match, etc. If None, pool headers are used. If provided, these headers completely replace any pool-specific headers. :param retries: Configure the number of retries to allow before raising a :class:`~urllib3.exceptions.MaxRetryError` exception. Pass ``None`` to retry until you receive a response. Pass a :class:`~urllib3.util.retry.Retry` object for fine-grained control over different types of retries. Pass an integer number to retry connection errors that many times, but no other types of errors. Pass zero to never retry. If ``False``, then retries are disabled and any exception is raised immediately. Also, instead of raising a MaxRetryError on redirects, the redirect response will be returned. :type retries: :class:`~urllib3.util.retry.Retry`, False, or an int. :param redirect: If True, automatically handle redirects (status codes 301, 302, 303, 307, 308). Each redirect counts as a retry. Disabling retries will disable redirect, too. :param assert_same_host: If ``True``, will make sure that the host of the pool requests is consistent else will raise HostChangedError. When ``False``, you can use the pool on an HTTP proxy and request foreign hosts. :param timeout: If specified, overrides the default timeout for this one request. It may be a float (in seconds) or an instance of :class:`urllib3.util.Timeout`. :param pool_timeout: If set and the pool is set to block=True, then this method will block for ``pool_timeout`` seconds and raise EmptyPoolError if no connection is available within the time period. :param release_conn: If False, then the urlopen call will not release the connection back into the pool once a response is received (but will release if you read the entire contents of the response such as when `preload_content=True`). This is useful if you're not preloading the response's content immediately. You will need to call ``r.release_conn()`` on the response ``r`` to return the connection back into the pool. If None, it takes the value of ``response_kw.get('preload_content', True)``. :param chunked: If True, urllib3 will send the body using chunked transfer encoding. Otherwise, urllib3 will send the body using the standard content-length form. Defaults to False. :param int body_pos: Position to seek to in file-like body in the event of a retry or redirect. Typically this won't need to be set because urllib3 will auto-populate the value when needed. :param \\**response_kw: Additional parameters are passed to :meth:`urllib3.response.HTTPResponse.from_httplib` """ parsed_url = parse_url(url) destination_scheme = parsed_url.scheme if headers is None: headers = self.headers if not isinstance(retries, Retry): retries = Retry.from_int(retries, redirect=redirect, default=self.retries) if release_conn is None: release_conn = response_kw.get("preload_content", True) # Check host if assert_same_host and not self.is_same_host(url): raise HostChangedError(self, url, retries) # Ensure that the URL we're connecting to is properly encoded if url.startswith("/"): url = six.ensure_str(_encode_target(url)) else: url = six.ensure_str(parsed_url.url) conn = None # Track whether `conn` needs to be released before # returning/raising/recursing. Update this variable if necessary, and # leave `release_conn` constant throughout the function. That way, if # the function recurses, the original value of `release_conn` will be # passed down into the recursive call, and its value will be respected. # # See issue #651 [1] for details. # # [1] <https://github.com/urllib3/urllib3/issues/651> release_this_conn = release_conn http_tunnel_required = connection_requires_http_tunnel( self.proxy, self.proxy_config, destination_scheme ) # Merge the proxy headers. Only done when not using HTTP CONNECT. We # have to copy the headers dict so we can safely change it without those # changes being reflected in anyone else's copy. if not http_tunnel_required: headers = headers.copy() headers.update(self.proxy_headers) # Must keep the exception bound to a separate variable or else Python 3 # complains about UnboundLocalError. err = None # Keep track of whether we cleanly exited the except block. This # ensures we do proper cleanup in finally. clean_exit = False # Rewind body position, if needed. Record current position # for future rewinds in the event of a redirect/retry. body_pos = set_file_position(body, body_pos) try: # Request a connection from the queue. timeout_obj = self._get_timeout(timeout) conn = self._get_conn(timeout=pool_timeout) conn.timeout = timeout_obj.connect_timeout is_new_proxy_conn = self.proxy is not None and not getattr( conn, "sock", None ) if is_new_proxy_conn and http_tunnel_required: self._prepare_proxy(conn) # Make the request on the httplib connection object. > httplib_response = self._make_request( conn, method, url, timeout=timeout_obj, body=body, headers=headers, chunked=chunked, ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:715: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff542148100> conn = <urllib3.connection.HTTPSConnection object at 0x7ff540d27070> method = 'POST', url = '/user/credentials/cdis/access_token' timeout = Timeout(connect=None, read=None, total=None), chunked = False httplib_request_kw = {'body': b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwd...p, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}} timeout_obj = Timeout(connect=None, read=None, total=None) def _make_request( self, conn, method, url, timeout=_Default, chunked=False, **httplib_request_kw ): """ Perform a request on a given urllib connection object taken from our pool. :param conn: a connection from one of our connection pools :param timeout: Socket timeout in seconds for the request. This can be a float or integer, which will set the same timeout value for the socket connect and the socket read, or an instance of :class:`urllib3.util.Timeout`, which gives you more fine-grained control over your timeouts. """ self.num_requests += 1 timeout_obj = self._get_timeout(timeout) timeout_obj.start_connect() conn.timeout = Timeout.resolve_default_timeout(timeout_obj.connect_timeout) # Trigger any extra validation we need to do. try: > self._validate_conn(conn) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:404: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff542148100> conn = <urllib3.connection.HTTPSConnection object at 0x7ff540d27070> def _validate_conn(self, conn): """ Called right before a request is made, after the socket is created. """ super(HTTPSConnectionPool, self)._validate_conn(conn) # Force connect early to allow us to validate the connection. if not getattr(conn, "sock", None): # AppEngine might not have `.sock` > conn.connect() ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:1058: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <urllib3.connection.HTTPSConnection object at 0x7ff540d27070> def connect(self): # Add certificate verification self.sock = conn = self._new_conn() hostname = self.host tls_in_tls = False if self._is_using_tunnel(): if self.tls_in_tls_required: self.sock = conn = self._connect_tls_proxy(hostname, conn) tls_in_tls = True # Calls self._set_hostport(), so self.host is # self._tunnel_host below. self._tunnel() # Mark this connection as not reusable self.auto_open = 0 # Override the host with the one we're requesting data from. hostname = self._tunnel_host server_hostname = hostname if self.server_hostname is not None: server_hostname = self.server_hostname is_time_off = datetime.date.today() < RECENT_DATE if is_time_off: warnings.warn( ( "System time is way off (before {0}). This will probably " "lead to SSL verification errors" ).format(RECENT_DATE), SystemTimeWarning, ) # Wrap socket using verification with the root certs in # trusted_root_certs default_ssl_context = False if self.ssl_context is None: default_ssl_context = True self.ssl_context = create_urllib3_context( ssl_version=resolve_ssl_version(self.ssl_version), cert_reqs=resolve_cert_reqs(self.cert_reqs), ) context = self.ssl_context context.verify_mode = resolve_cert_reqs(self.cert_reqs) # Try to load OS default certs if none are given. # Works well on Windows (requires Python3.4+) if ( not self.ca_certs and not self.ca_cert_dir and not self.ca_cert_data and default_ssl_context and hasattr(context, "load_default_certs") ): context.load_default_certs() > self.sock = ssl_wrap_socket( sock=conn, keyfile=self.key_file, certfile=self.cert_file, key_password=self.key_password, ca_certs=self.ca_certs, ca_cert_dir=self.ca_cert_dir, ca_cert_data=self.ca_cert_data, server_hostname=server_hostname, ssl_context=context, tls_in_tls=tls_in_tls, ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:419: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> keyfile = None, certfile = None, cert_reqs = None ca_certs = '/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/certifi/cacert.pem' server_hostname = 'gen3.abi-ctt-ctp.cloud.edu.au', ssl_version = None ciphers = None, ssl_context = <ssl.SSLContext object at 0x7ff540e27c40> ca_cert_dir = None, key_password = None, ca_cert_data = None, tls_in_tls = False def ssl_wrap_socket( sock, keyfile=None, certfile=None, cert_reqs=None, ca_certs=None, server_hostname=None, ssl_version=None, ciphers=None, ssl_context=None, ca_cert_dir=None, key_password=None, ca_cert_data=None, tls_in_tls=False, ): """ All arguments except for server_hostname, ssl_context, and ca_cert_dir have the same meaning as they do when using :func:`ssl.wrap_socket`. :param server_hostname: When SNI is supported, the expected hostname of the certificate :param ssl_context: A pre-made :class:`SSLContext` object. If none is provided, one will be created using :func:`create_urllib3_context`. :param ciphers: A string of ciphers we wish the client to support. :param ca_cert_dir: A directory containing CA certificates in multiple separate files, as supported by OpenSSL's -CApath flag or the capath argument to SSLContext.load_verify_locations(). :param key_password: Optional password if the keyfile is encrypted. :param ca_cert_data: Optional string containing CA certificates in PEM format suitable for passing as the cadata parameter to SSLContext.load_verify_locations() :param tls_in_tls: Use SSLTransport to wrap the existing socket. """ context = ssl_context if context is None: # Note: This branch of code and all the variables in it are no longer # used by urllib3 itself. We should consider deprecating and removing # this code. context = create_urllib3_context(ssl_version, cert_reqs, ciphers=ciphers) if ca_certs or ca_cert_dir or ca_cert_data: try: context.load_verify_locations(ca_certs, ca_cert_dir, ca_cert_data) except (IOError, OSError) as e: raise SSLError(e) elif ssl_context is None and hasattr(context, "load_default_certs"): # try to load OS default certs; works well on Windows (require Python3.4+) context.load_default_certs() # Attempt to detect if we get the goofy behavior of the # keyfile being encrypted and OpenSSL asking for the # passphrase via the terminal and instead error out. if keyfile and key_password is None and _is_key_file_encrypted(keyfile): raise SSLError("Client private key is encrypted, password is required") if certfile: if key_password is None: context.load_cert_chain(certfile, keyfile) else: context.load_cert_chain(certfile, keyfile, key_password) try: if hasattr(context, "set_alpn_protocols"): context.set_alpn_protocols(ALPN_PROTOCOLS) except NotImplementedError: # Defensive: in CI, we always have set_alpn_protocols pass # If we detect server_hostname is an IP address then the SNI # extension should not be used according to RFC3546 Section 3.1 use_sni_hostname = server_hostname and not is_ipaddress(server_hostname) # SecureTransport uses server_hostname in certificate verification. send_sni = (use_sni_hostname and HAS_SNI) or ( IS_SECURETRANSPORT and server_hostname ) # Do not warn the user if server_hostname is an invalid SNI hostname. if not HAS_SNI and use_sni_hostname: warnings.warn( "An HTTPS request has been made, but the SNI (Server Name " "Indication) extension to TLS is not available on this platform. " "This may cause the server to present an incorrect TLS " "certificate, which can cause validation failures. You can upgrade to " "a newer version of Python to solve this. For more information, see " "https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html" "#ssl-warnings", SNIMissingWarning, ) if send_sni: > ssl_sock = _ssl_wrap_socket_impl( sock, context, tls_in_tls, server_hostname=server_hostname ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/ssl_.py:449: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> ssl_context = <ssl.SSLContext object at 0x7ff540e27c40>, tls_in_tls = False server_hostname = 'gen3.abi-ctt-ctp.cloud.edu.au' def _ssl_wrap_socket_impl(sock, ssl_context, tls_in_tls, server_hostname=None): if tls_in_tls: if not SSLTransport: # Import error, ssl is not available. raise ProxySchemeUnsupported( "TLS in TLS requires support for the 'ssl' module" ) SSLTransport._validate_ssl_context_for_tls_in_tls(ssl_context) return SSLTransport(sock, ssl_context, server_hostname) if server_hostname: > return ssl_context.wrap_socket(sock, server_hostname=server_hostname) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/ssl_.py:493: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <ssl.SSLContext object at 0x7ff540e27c40> sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> server_side = False, do_handshake_on_connect = True, suppress_ragged_eofs = True server_hostname = 'gen3.abi-ctt-ctp.cloud.edu.au', session = None def wrap_socket(self, sock, server_side=False, do_handshake_on_connect=True, suppress_ragged_eofs=True, server_hostname=None, session=None): # SSLSocket class handles server_hostname encoding before it calls # ctx._wrap_socket() > return self.sslsocket_class._create( sock=sock, server_side=server_side, do_handshake_on_connect=do_handshake_on_connect, suppress_ragged_eofs=suppress_ragged_eofs, server_hostname=server_hostname, context=self, session=session ) /usr/lib/python3.9/ssl.py:501: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ cls = <class 'ssl.SSLSocket'> sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> server_side = False, do_handshake_on_connect = True, suppress_ragged_eofs = True server_hostname = 'gen3.abi-ctt-ctp.cloud.edu.au' context = <ssl.SSLContext object at 0x7ff540e27c40>, session = None @classmethod def _create(cls, sock, server_side=False, do_handshake_on_connect=True, suppress_ragged_eofs=True, server_hostname=None, context=None, session=None): if sock.getsockopt(SOL_SOCKET, SO_TYPE) != SOCK_STREAM: raise NotImplementedError("only stream sockets are supported") if server_side: if server_hostname: raise ValueError("server_hostname can only be specified " "in client mode") if session is not None: raise ValueError("session can only be specified in " "client mode") if context.check_hostname and not server_hostname: raise ValueError("check_hostname requires server_hostname") kwargs = dict( family=sock.family, type=sock.type, proto=sock.proto, fileno=sock.fileno() ) self = cls.__new__(cls, **kwargs) super(SSLSocket, self).__init__(**kwargs) sock_timeout = sock.gettimeout() sock.detach() self._context = context self._session = session self._closed = False self._sslobj = None self.server_side = server_side self.server_hostname = context._encode_hostname(server_hostname) self.do_handshake_on_connect = do_handshake_on_connect self.suppress_ragged_eofs = suppress_ragged_eofs # See if we are connected try: self.getpeername() except OSError as e: if e.errno != errno.ENOTCONN: raise connected = False blocking = self.getblocking() self.setblocking(False) try: # We are not connected so this is not supposed to block, but # testing revealed otherwise on macOS and Windows so we do # the non-blocking dance regardless. Our raise when any data # is found means consuming the data is harmless. notconn_pre_handshake_data = self.recv(1) except OSError as e: # EINVAL occurs for recv(1) on non-connected on unix sockets. if e.errno not in (errno.ENOTCONN, errno.EINVAL): raise notconn_pre_handshake_data = b'' self.setblocking(blocking) if notconn_pre_handshake_data: # This prevents pending data sent to the socket before it was # closed from escaping to the caller who could otherwise # presume it came through a successful TLS connection. reason = "Closed before TLS handshake with data in recv buffer." notconn_pre_handshake_data_error = SSLError(e.errno, reason) # Add the SSLError attributes that _ssl.c always adds. notconn_pre_handshake_data_error.reason = reason notconn_pre_handshake_data_error.library = None try: self.close() except OSError: pass try: raise notconn_pre_handshake_data_error finally: # Explicitly break the reference cycle. notconn_pre_handshake_data_error = None else: connected = True self.settimeout(sock_timeout) # Must come after setblocking() calls. self._connected = connected if connected: # create the SSL object try: self._sslobj = self._context._wrap_socket( self, server_side, self.server_hostname, owner=self, session=self._session, ) if do_handshake_on_connect: timeout = self.gettimeout() if timeout == 0.0: # non-blocking raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets") > self.do_handshake() /usr/lib/python3.9/ssl.py:1074: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <ssl.SSLSocket [closed] fd=-1, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6> block = False @_sslcopydoc def do_handshake(self, block=False): self._check_connected() timeout = self.gettimeout() try: if timeout == 0.0 and block: self.settimeout(None) > self._sslobj.do_handshake() E ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133) /usr/lib/python3.9/ssl.py:1343: SSLCertVerificationError During handling of the above exception, another exception occurred: self = <requests.adapters.HTTPAdapter object at 0x7ff540a40c10> request = <PreparedRequest [POST]>, stream = False timeout = Timeout(connect=None, read=None, total=None), verify = True cert = None, proxies = OrderedDict() def send( self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None ): """Sends PreparedRequest object. Returns Response object. :param request: The :class:`PreparedRequest <PreparedRequest>` being sent. :param stream: (optional) Whether to stream the request content. :param timeout: (optional) How long to wait for the server to send data before giving up, as a float, or a :ref:`(connect timeout, read timeout) <timeouts>` tuple. :type timeout: float or tuple or urllib3 Timeout object :param verify: (optional) Either a boolean, in which case it controls whether we verify the server's TLS certificate, or a string, in which case it must be a path to a CA bundle to use :param cert: (optional) Any user-provided SSL certificate to be trusted. :param proxies: (optional) The proxies dictionary to apply to the request. :rtype: requests.Response """ try: conn = self.get_connection(request.url, proxies) except LocationValueError as e: raise InvalidURL(e, request=request) self.cert_verify(conn, request.url, verify, cert) url = self.request_url(request, proxies) self.add_headers( request, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies, ) chunked = not (request.body is None or "Content-Length" in request.headers) if isinstance(timeout, tuple): try: connect, read = timeout timeout = TimeoutSauce(connect=connect, read=read) except ValueError: raise ValueError( f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, " f"or a single float to set both timeouts to the same value." ) elif isinstance(timeout, TimeoutSauce): pass else: timeout = TimeoutSauce(connect=timeout, read=timeout) try: > resp = conn.urlopen( method=request.method, url=url, body=request.body, headers=request.headers, redirect=False, assert_same_host=False, preload_content=False, decode_content=False, retries=self.max_retries, timeout=timeout, chunked=chunked, ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:486: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff542148100> method = 'POST', url = '/user/credentials/cdis/access_token' body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...NPva0i6TdvsONgWYV1N3mET3cX78Px5n6SH84sTMCLqWr5EOzw4ImhcQ2kOYfMOqKA", "key_id": "92c93027-24d6-4240-825c-30b9395d78cf"}' headers = {'User-Agent': 'python-requests/2.31.0', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'} retries = Retry(total=0, connect=None, read=False, redirect=None, status=None) redirect = False, assert_same_host = False timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None release_conn = False, chunked = False, body_pos = None response_kw = {'decode_content': False, 'preload_content': False} parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None) destination_scheme = None, conn = None, release_this_conn = True http_tunnel_required = False, err = None, clean_exit = False def urlopen( self, method, url, body=None, headers=None, retries=None, redirect=True, assert_same_host=True, timeout=_Default, pool_timeout=None, release_conn=None, chunked=False, body_pos=None, **response_kw ): """ Get a connection from the pool and perform an HTTP request. This is the lowest level call for making a request, so you'll need to specify all the raw details. .. note:: More commonly, it's appropriate to use a convenience method provided by :class:`.RequestMethods`, such as :meth:`request`. .. note:: `release_conn` will only behave as expected if `preload_content=False` because we want to make `preload_content=False` the default behaviour someday soon without breaking backwards compatibility. :param method: HTTP request method (such as GET, POST, PUT, etc.) :param url: The URL to perform the request on. :param body: Data to send in the request body, either :class:`str`, :class:`bytes`, an iterable of :class:`str`/:class:`bytes`, or a file-like object. :param headers: Dictionary of custom headers to send, such as User-Agent, If-None-Match, etc. If None, pool headers are used. If provided, these headers completely replace any pool-specific headers. :param retries: Configure the number of retries to allow before raising a :class:`~urllib3.exceptions.MaxRetryError` exception. Pass ``None`` to retry until you receive a response. Pass a :class:`~urllib3.util.retry.Retry` object for fine-grained control over different types of retries. Pass an integer number to retry connection errors that many times, but no other types of errors. Pass zero to never retry. If ``False``, then retries are disabled and any exception is raised immediately. Also, instead of raising a MaxRetryError on redirects, the redirect response will be returned. :type retries: :class:`~urllib3.util.retry.Retry`, False, or an int. :param redirect: If True, automatically handle redirects (status codes 301, 302, 303, 307, 308). Each redirect counts as a retry. Disabling retries will disable redirect, too. :param assert_same_host: If ``True``, will make sure that the host of the pool requests is consistent else will raise HostChangedError. When ``False``, you can use the pool on an HTTP proxy and request foreign hosts. :param timeout: If specified, overrides the default timeout for this one request. It may be a float (in seconds) or an instance of :class:`urllib3.util.Timeout`. :param pool_timeout: If set and the pool is set to block=True, then this method will block for ``pool_timeout`` seconds and raise EmptyPoolError if no connection is available within the time period. :param release_conn: If False, then the urlopen call will not release the connection back into the pool once a response is received (but will release if you read the entire contents of the response such as when `preload_content=True`). This is useful if you're not preloading the response's content immediately. You will need to call ``r.release_conn()`` on the response ``r`` to return the connection back into the pool. If None, it takes the value of ``response_kw.get('preload_content', True)``. :param chunked: If True, urllib3 will send the body using chunked transfer encoding. Otherwise, urllib3 will send the body using the standard content-length form. Defaults to False. :param int body_pos: Position to seek to in file-like body in the event of a retry or redirect. Typically this won't need to be set because urllib3 will auto-populate the value when needed. :param \\**response_kw: Additional parameters are passed to :meth:`urllib3.response.HTTPResponse.from_httplib` """ parsed_url = parse_url(url) destination_scheme = parsed_url.scheme if headers is None: headers = self.headers if not isinstance(retries, Retry): retries = Retry.from_int(retries, redirect=redirect, default=self.retries) if release_conn is None: release_conn = response_kw.get("preload_content", True) # Check host if assert_same_host and not self.is_same_host(url): raise HostChangedError(self, url, retries) # Ensure that the URL we're connecting to is properly encoded if url.startswith("/"): url = six.ensure_str(_encode_target(url)) else: url = six.ensure_str(parsed_url.url) conn = None # Track whether `conn` needs to be released before # returning/raising/recursing. Update this variable if necessary, and # leave `release_conn` constant throughout the function. That way, if # the function recurses, the original value of `release_conn` will be # passed down into the recursive call, and its value will be respected. # # See issue #651 [1] for details. # # [1] <https://github.com/urllib3/urllib3/issues/651> release_this_conn = release_conn http_tunnel_required = connection_requires_http_tunnel( self.proxy, self.proxy_config, destination_scheme ) # Merge the proxy headers. Only done when not using HTTP CONNECT. We # have to copy the headers dict so we can safely change it without those # changes being reflected in anyone else's copy. if not http_tunnel_required: headers = headers.copy() headers.update(self.proxy_headers) # Must keep the exception bound to a separate variable or else Python 3 # complains about UnboundLocalError. err = None # Keep track of whether we cleanly exited the except block. This # ensures we do proper cleanup in finally. clean_exit = False # Rewind body position, if needed. Record current position # for future rewinds in the event of a redirect/retry. body_pos = set_file_position(body, body_pos) try: # Request a connection from the queue. timeout_obj = self._get_timeout(timeout) conn = self._get_conn(timeout=pool_timeout) conn.timeout = timeout_obj.connect_timeout is_new_proxy_conn = self.proxy is not None and not getattr( conn, "sock", None ) if is_new_proxy_conn and http_tunnel_required: self._prepare_proxy(conn) # Make the request on the httplib connection object. httplib_response = self._make_request( conn, method, url, timeout=timeout_obj, body=body, headers=headers, chunked=chunked, ) # If we're going to release the connection in ``finally:``, then # the response doesn't need to know about the connection. Otherwise # it will also try to release it and we'll have a double-release # mess. response_conn = conn if not release_conn else None # Pass method to Response for length checking response_kw["request_method"] = method # Import httplib's response into our own wrapper object response = self.ResponseCls.from_httplib( httplib_response, pool=self, connection=response_conn, retries=retries, **response_kw ) # Everything went great! clean_exit = True except EmptyPoolError: # Didn't get a connection from the pool, no need to clean up clean_exit = True release_this_conn = False raise except ( TimeoutError, HTTPException, SocketError, ProtocolError, BaseSSLError, SSLError, CertificateError, ) as e: # Discard the connection for these exceptions. It will be # replaced during the next _get_conn() call. clean_exit = False def _is_ssl_error_message_from_http_proxy(ssl_error): # We're trying to detect the message 'WRONG_VERSION_NUMBER' but # SSLErrors are kinda all over the place when it comes to the message, # so we try to cover our bases here! message = " ".join(re.split("[^a-z]", str(ssl_error).lower())) return ( "wrong version number" in message or "unknown protocol" in message ) # Try to detect a common user error with proxies which is to # set an HTTP proxy to be HTTPS when it should be 'http://' # (ie {'http': 'http://proxy', 'https': 'https://proxy'}) # Instead we add a nice error message and point to a URL. if ( isinstance(e, BaseSSLError) and self.proxy and _is_ssl_error_message_from_http_proxy(e) and conn.proxy and conn.proxy.scheme == "https" ): e = ProxyError( "Your proxy appears to only use HTTP and not HTTPS, " "try changing your proxy URL to be HTTP. See: " "https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html" "#https-proxy-error-http-proxy", SSLError(e), ) elif isinstance(e, (BaseSSLError, CertificateError)): e = SSLError(e) elif isinstance(e, (SocketError, NewConnectionError)) and self.proxy: e = ProxyError("Cannot connect to proxy.", e) elif isinstance(e, (SocketError, HTTPException)): e = ProtocolError("Connection aborted.", e) > retries = retries.increment( method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] ) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:799: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = Retry(total=0, connect=None, read=False, redirect=None, status=None) method = 'POST', url = '/user/credentials/cdis/access_token', response = None error = SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133)')) _pool = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7ff542148100> _stacktrace = <traceback object at 0x7ff54b482940> def increment( self, method=None, url=None, response=None, error=None, _pool=None, _stacktrace=None, ): """Return a new Retry object with incremented retry counters. :param response: A response object, or None, if the server did not return a response. :type response: :class:`~urllib3.response.HTTPResponse` :param Exception error: An error encountered during the request, or None if the response was received successfully. :return: A new ``Retry`` object. """ if self.total is False and error: # Disabled, indicate to re-raise the error. raise six.reraise(type(error), error, _stacktrace) total = self.total if total is not None: total -= 1 connect = self.connect read = self.read redirect = self.redirect status_count = self.status other = self.other cause = "unknown" status = None redirect_location = None if error and self._is_connection_error(error): # Connect retry? if connect is False: raise six.reraise(type(error), error, _stacktrace) elif connect is not None: connect -= 1 elif error and self._is_read_error(error): # Read retry? if read is False or not self._is_method_retryable(method): raise six.reraise(type(error), error, _stacktrace) elif read is not None: read -= 1 elif error: # Other retry? if other is not None: other -= 1 elif response and response.get_redirect_location(): # Redirect retry? if redirect is not None: redirect -= 1 cause = "too many redirects" redirect_location = response.get_redirect_location() status = response.status else: # Incrementing because of a server error like a 500 in # status_forcelist and the given method is in the allowed_methods cause = ResponseError.GENERIC_ERROR if response and response.status: if status_count is not None: status_count -= 1 cause = ResponseError.SPECIFIC_ERROR.format(status_code=response.status) status = response.status history = self.history + ( RequestHistory(method, url, error, status, redirect_location), ) new_retry = self.new( total=total, connect=connect, read=read, redirect=redirect, status=status_count, other=other, history=history, ) if new_retry.is_exhausted(): > raise MaxRetryError(_pool, url, error or ResponseError(cause)) E urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133)'))) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/retry.py:592: MaxRetryError During handling of the above exception, another exception occurred: client = <starlette.testclient.TestClient object at 0x7ff540654970> token = {'access_token': 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZGVudGl0eSI6InB1YmxpYyIsInNjb3BlIjpbImRlbW8xLTEyTCJdLCJleHBpcmUiOiJOb25lIn0.doEgWg8P2u6dGhajC1CBwJHS3TtbJh4L_vhMfK75uJQ'} def test_case_node(client, token): UUID = "c58ab983-6cf9-4174-a7a9-20cdf1d6bc33" > response = client.get( f"/record/{UUID}", headers={"Authorization": f"Bearer {token['access_token']}"}, ) tests/test_data/test_gen3_data.py:227: _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:602: in get return self.request("GET", url, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:476: in request return super().request( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:589: in request resp = self.send(prep, **send_kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:703: in send r = adapter.send(request, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:270: in send raise exc ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:267: in send portal.call(self.app, scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:283: in call return cast(T_Retval, self.start_task_soon(func, *args).result()) /usr/lib/python3.9/concurrent/futures/_base.py:446: in result return self.__get_result() /usr/lib/python3.9/concurrent/futures/_base.py:391: in __get_result raise self._exception ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:219: in _call_func retval = await retval ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/applications.py:269: in __call__ await super().__call__(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/applications.py:124: in __call__ await self.middleware_stack(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:184: in __call__ raise exc ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:162: in __call__ await self.app(scope, receive, _send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/cors.py:84: in __call__ await self.app(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:93: in __call__ raise exc ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:82: in __call__ await self.app(scope, receive, sender) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:21: in __call__ raise e ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:18: in __call__ await self.app(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:670: in __call__ await route.handle(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:266: in handle await self.app(scope, receive, send) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:65: in app response = await func(request) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:227: in app raw_response = await run_endpoint_function( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:160: in run_endpoint_function return await dependant.call(**values) app/main.py:318: in get_gen3_record record = connection["gen3"].export_record(program, project, uuid, "json") ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/submission.py:348: in export_record output = requests.get(api_url, auth=self._auth_provider).text ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:73: in get return request("get", url, params=params, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request return session.request(method=method, url=url, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:575: in request prep = self.prepare_request(req) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:486: in prepare_request p.prepare( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:372: in prepare self.prepare_auth(auth, url) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:603: in prepare_auth r = auth(self) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:325: in __call__ request.headers["Authorization"] = self._get_auth_value() ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:446: in _get_auth_value return "bearer " + self.get_access_token() ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:433: in get_access_token return self.refresh_access_token( ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:367: in refresh_access_token self._access_token = get_access_token_with_key(self._refresh_token) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:75: in get_access_token_with_key resp = requests.post(auth_url, json=api_key) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:115: in post return request("post", url, data=data, json=json, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request return session.request(method=method, url=url, **kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:589: in request resp = self.send(prep, **send_kwargs) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:703: in send r = adapter.send(request, **kwargs) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ self = <requests.adapters.HTTPAdapter object at 0x7ff540a40c10> request = <PreparedRequest [POST]>, stream = False timeout = Timeout(connect=None, read=None, total=None), verify = True cert = None, proxies = OrderedDict() def send( self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None ): """Sends PreparedRequest object. Returns Response object. :param request: The :class:`PreparedRequest <PreparedRequest>` being sent. :param stream: (optional) Whether to stream the request content. :param timeout: (optional) How long to wait for the server to send data before giving up, as a float, or a :ref:`(connect timeout, read timeout) <timeouts>` tuple. :type timeout: float or tuple or urllib3 Timeout object :param verify: (optional) Either a boolean, in which case it controls whether we verify the server's TLS certificate, or a string, in which case it must be a path to a CA bundle to use :param cert: (optional) Any user-provided SSL certificate to be trusted. :param proxies: (optional) The proxies dictionary to apply to the request. :rtype: requests.Response """ try: conn = self.get_connection(request.url, proxies) except LocationValueError as e: raise InvalidURL(e, request=request) self.cert_verify(conn, request.url, verify, cert) url = self.request_url(request, proxies) self.add_headers( request, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies, ) chunked = not (request.body is None or "Content-Length" in request.headers) if isinstance(timeout, tuple): try: connect, read = timeout timeout = TimeoutSauce(connect=connect, read=read) except ValueError: raise ValueError( f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, " f"or a single float to set both timeouts to the same value." ) elif isinstance(timeout, TimeoutSauce): pass else: timeout = TimeoutSauce(connect=timeout, read=timeout) try: resp = conn.urlopen( method=request.method, url=url, body=request.body, headers=request.headers, redirect=False, assert_same_host=False, preload_content=False, decode_content=False, retries=self.max_retries, timeout=timeout, chunked=chunked, ) except (ProtocolError, OSError) as err: raise ConnectionError(err, request=request) except MaxRetryError as e: if isinstance(e.reason, ConnectTimeoutError): # TODO: Remove this in 3.0.0: see #2811 if not isinstance(e.reason, NewConnectionError): raise ConnectTimeout(e, request=request) if isinstance(e.reason, ResponseError): raise RetryError(e, request=request) if isinstance(e.reason, _ProxyError): raise ProxyError(e, request=request) if isinstance(e.reason, _SSLError): # This branch is for urllib3 v1.22 and later. > raise SSLError(e, request=request) E requests.exceptions.SSLError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133)'))) ../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:517: SSLError ------------------------------ Captured log setup ------------------------------ ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. INFO app.main:main.py:166 {'gen3': False, 'irods': True, 'orthanc': True} WARNING app.main:main.py:185 Failed to update default filter. ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. ------------------------------ Captured log call ------------------------------- ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. ____________________________ test_irods_collection _____________________________ client = <starlette.testclient.TestClient object at 0x7ff5400e95b0> token = {'access_token': 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZGVudGl0eSI6InB1YmxpYyIsInNjb3BlIjpbImRlbW8xLTEyTCJdLCJleHBpcmUiOiJOb25lIn0.doEgWg8P2u6dGhajC1CBwJHS3TtbJh4L_vhMfK75uJQ'} def test_irods_collection(client, token): payload1 = {"path": "/dataset-217-version-2"} response = client.post( "/collection", json=payload1, headers={"Authorization": f"Bearer {token['access_token']}"}, ) result = response.json() > assert result["folders"] == [ {"name": "derivative", "path": "/dataset-217-version-2/derivative"}, {"name": "primary", "path": "/dataset-217-version-2/primary"}, ] E KeyError: 'folders' tests/test_data/test_irods_data.py:32: KeyError ------------------------------ Captured log setup ------------------------------ ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. INFO app.main:main.py:166 {'gen3': False, 'irods': True, 'orthanc': True} WARNING app.main:main.py:185 Failed to update default filter. ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. ------------------------------ Captured log call ------------------------------- ERROR services.gen3.gen3_service:gen3_service.py:128 Failed to create the Gen3 submission. =============================== warnings summary =============================== tests/test_api/test_gen3_api.py::test_get_gen3_graphql_query /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/_pytest/threadexception.py:73: PytestUnhandledThreadExceptionWarning: Exception in thread Thread-1 Traceback (most recent call last): File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py", line 715, in urlopen httplib_response = self._make_request( File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py", line 404, in _make_request self._validate_conn(conn) File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py", line 1058, in _validate_conn conn.connect() File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py", line 419, in connect self.sock = ssl_wrap_socket( File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 449, in ssl_wrap_socket ssl_sock = _ssl_wrap_socket_impl( File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/ssl_.py", line 493, in _ssl_wrap_socket_impl return ssl_context.wrap_socket(sock, server_hostname=server_hostname) File "/usr/lib/python3.9/ssl.py", line 501, in wrap_socket return self.sslsocket_class._create( File "/usr/lib/python3.9/ssl.py", line 1074, in _create self.do_handshake() File "/usr/lib/python3.9/ssl.py", line 1343, in do_handshake self._sslobj.do_handshake() ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py", line 486, in send resp = conn.urlopen( File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py", line 799, in urlopen retries = retries.increment( File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/retry.py", line 592, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133)'))) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/cmiss/Jenkins/workspace/12-Labours-API/services/gen3/gen3_service.py", line 43, in process_graphql_query query_result = self.__submission.query(query_code)["data"][item.node] File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/submission.py", line 423, in query output = requests.post(api_url, auth=self._auth_provider, json=query).text File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py", line 115, in post return request("post", url, data=data, json=json, **kwargs) File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py", line 59, in request return session.request(method=method, url=url, **kwargs) File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py", line 575, in request prep = self.prepare_request(req) File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py", line 486, in prepare_request p.prepare( File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py", line 372, in prepare self.prepare_auth(auth, url) File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py", line 603, in prepare_auth r = auth(self) File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py", line 325, in __call__ request.headers["Authorization"] = self._get_auth_value() File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py", line 446, in _get_auth_value return "bearer " + self.get_access_token() File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py", line 433, in get_access_token return self.refresh_access_token( File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py", line 367, in refresh_access_token self._access_token = get_access_token_with_key(self._refresh_token) File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py", line 75, in get_access_token_with_key resp = requests.post(auth_url, json=api_key) File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py", line 115, in post return request("post", url, data=data, json=json, **kwargs) File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py", line 59, in request return session.request(method=method, url=url, **kwargs) File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, **send_kwargs) File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, **kwargs) File "/home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py", line 517, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: certificate has expired (_ssl.c:1133)'))) The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/lib/python3.9/threading.py", line 980, in _bootstrap_inner self.run() File "/usr/lib/python3.9/threading.py", line 917, in run self._target(*self._args, **self._kwargs) File "/home/cmiss/Jenkins/workspace/12-Labours-API/services/gen3/gen3_service.py", line 48, in process_graphql_query raise HTTPException( fastapi.exceptions.HTTPException warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg)) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html =========================== short test summary info ============================ FAILED tests/test_api/test_gen3_api.py::test_get_gen3_record - requests.excep... FAILED tests/test_api/test_gen3_api.py::test_get_gen3_graphql_query - KeyErro... FAILED tests/test_api/test_gen3_api.py::test_get_gen3_graphql_pagination - as... FAILED tests/test_api/test_irods_api.py::test_get_irods_collection - assert 4... FAILED tests/test_api/test_irods_api.py::test_get_irods_data_file - assert 40... FAILED tests/test_data/test_gen3_data.py::test_experiment_node - requests.exc... FAILED tests/test_data/test_gen3_data.py::test_dataset_description_node - req... FAILED tests/test_data/test_gen3_data.py::test_manifest_node - requests.excep... FAILED tests/test_data/test_gen3_data.py::test_case_node - requests.exception... FAILED tests/test_data/test_irods_data.py::test_irods_collection - KeyError: ... ============= 10 failed, 29 passed, 1 warning in 166.18s (0:02:46) ============= Build step 'Virtualenv Builder' marked build as failure [Slack Notifications] found #2654 as previous completed, non-aborted build [Slack Notifications] will send OnEveryFailureNotification because build matches and user preferences allow it Finished: FAILURE