Started by timer
Running as SYSTEM
Building remotely on Ubuntu_18.04_bioeng49 (buildslave Testing) in workspace /home/cmiss/Jenkins/workspace/12-Labours-API
[WS-CLEANUP] Deleting project workspace...
[WS-CLEANUP] Deferred wipeout is used...
[WS-CLEANUP] Done
The recommended git tool is: NONE
No credentials specified
Cloning the remote Git repository
Cloning repository https://github.com/ABI-Software/12-labours-api.git
> git init /home/cmiss/Jenkins/workspace/12-Labours-API # timeout=10
Fetching upstream changes from https://github.com/ABI-Software/12-labours-api.git
> git --version # timeout=10
> git --version # 'git version 2.25.1'
> git fetch --tags --force --progress -- https://github.com/ABI-Software/12-labours-api.git +refs/heads/*:refs/remotes/origin/* # timeout=10
> git config remote.origin.url https://github.com/ABI-Software/12-labours-api.git # timeout=10
> git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
Avoid second fetch
> git rev-parse refs/remotes/origin/dev^{commit} # timeout=10
Checking out Revision 6845b092c43983ec60d36bc431776576457e354b (refs/remotes/origin/dev)
> git config core.sparsecheckout # timeout=10
> git checkout -f 6845b092c43983ec60d36bc431776576457e354b # timeout=10
Commit message: "Merge pull request #18 from david-jianwei-yu/query-used-sgqlc-schema-update"
> git rev-list --no-walk 6845b092c43983ec60d36bc431776576457e354b # timeout=10
[12-Labours-API] $ /bin/sh -xe /tmp/shiningpanda4987760701665477949.sh
+ pwd
+ export PYTHONPATH=/home/cmiss/Jenkins/workspace/12-Labours-API
+ pip install -r requirements.txt
Requirement already satisfied: autopep8==1.6.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements.txt (line 1)) (1.6.0)
Requirement already satisfied: fastapi==0.80.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements.txt (line 2)) (0.80.0)
Requirement already satisfied: gen3==4.15.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements.txt (line 3)) (4.15.0)
Requirement already satisfied: gunicorn==20.1.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements.txt (line 4)) (20.1.0)
Requirement already satisfied: python-dotenv==0.20.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements.txt (line 5)) (0.20.0)
Requirement already satisfied: python-irodsclient==1.1.4 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements.txt (line 6)) (1.1.4)
Requirement already satisfied: sgqlc==16.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements.txt (line 7)) (16.0)
Requirement already satisfied: uvicorn==0.18.2 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements.txt (line 8)) (0.18.2)
Requirement already satisfied: pycodestyle>=2.8.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from autopep8==1.6.0->-r requirements.txt (line 1)) (2.10.0)
Requirement already satisfied: toml in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from autopep8==1.6.0->-r requirements.txt (line 1)) (0.10.2)
Requirement already satisfied: pydantic!=1.7,!=1.7.1,!=1.7.2,!=1.7.3,!=1.8,!=1.8.1,<2.0.0,>=1.6.2 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from fastapi==0.80.0->-r requirements.txt (line 2)) (1.10.4)
Requirement already satisfied: starlette==0.19.1 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from fastapi==0.80.0->-r requirements.txt (line 2)) (0.19.1)
Requirement already satisfied: requests in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.15.0->-r requirements.txt (line 3)) (2.28.2)
Requirement already satisfied: tqdm<5.0.0,>=4.61.2 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.15.0->-r requirements.txt (line 3)) (4.64.1)
Requirement already satisfied: cdislogging<2.0.0,>=1.1.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.15.0->-r requirements.txt (line 3)) (1.1.1)
Requirement already satisfied: click in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.15.0->-r requirements.txt (line 3)) (7.1.2)
Requirement already satisfied: python-dateutil in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.15.0->-r requirements.txt (line 3)) (2.8.2)
Requirement already satisfied: pypfb<1.0.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.15.0->-r requirements.txt (line 3)) (0.5.21)
Requirement already satisfied: aiohttp in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.15.0->-r requirements.txt (line 3)) (3.8.3)
Requirement already satisfied: aiofiles<0.9.0,>=0.8.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.15.0->-r requirements.txt (line 3)) (0.8.0)
Requirement already satisfied: dataclasses-json in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.15.0->-r requirements.txt (line 3)) (0.5.7)
Requirement already satisfied: drsclient<0.3.0,>=0.2.2 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.15.0->-r requirements.txt (line 3)) (0.2.3)
Requirement already satisfied: jsonschema in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.15.0->-r requirements.txt (line 3)) (3.2.0)
Requirement already satisfied: pandas<2.0.0,>=1.4.2 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.15.0->-r requirements.txt (line 3)) (1.5.3)
Requirement already satisfied: httpx in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.15.0->-r requirements.txt (line 3)) (0.23.3)
Requirement already satisfied: humanfriendly in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.15.0->-r requirements.txt (line 3)) (10.0)
Requirement already satisfied: backoff in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.15.0->-r requirements.txt (line 3)) (1.11.1)
Requirement already satisfied: indexclient in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gen3==4.15.0->-r requirements.txt (line 3)) (2.1.0)
Requirement already satisfied: setuptools>=3.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from gunicorn==20.1.0->-r requirements.txt (line 4)) (66.1.1)
Requirement already satisfied: PrettyTable>=0.7.2 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from python-irodsclient==1.1.4->-r requirements.txt (line 6)) (3.6.0)
Requirement already satisfied: defusedxml in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from python-irodsclient==1.1.4->-r requirements.txt (line 6)) (0.7.1)
Requirement already satisfied: six>=1.10.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from python-irodsclient==1.1.4->-r requirements.txt (line 6)) (1.16.0)
Requirement already satisfied: graphql-core in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from sgqlc==16.0->-r requirements.txt (line 7)) (3.2.3)
Requirement already satisfied: h11>=0.8 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from uvicorn==0.18.2->-r requirements.txt (line 8)) (0.14.0)
Requirement already satisfied: typing-extensions>=3.10.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from starlette==0.19.1->fastapi==0.80.0->-r requirements.txt (line 2)) (4.4.0)
Requirement already satisfied: anyio<5,>=3.4.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from starlette==0.19.1->fastapi==0.80.0->-r requirements.txt (line 2)) (3.6.2)
Requirement already satisfied: asyncio<4.0.0,>=3.4.3 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from drsclient<0.3.0,>=0.2.2->gen3==4.15.0->-r requirements.txt (line 3)) (3.4.3)
Requirement already satisfied: rfc3986[idna2008]<2,>=1.3 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from httpx->gen3==4.15.0->-r requirements.txt (line 3)) (1.5.0)
Requirement already satisfied: sniffio in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from httpx->gen3==4.15.0->-r requirements.txt (line 3)) (1.3.0)
Requirement already satisfied: certifi in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from httpx->gen3==4.15.0->-r requirements.txt (line 3)) (2022.12.7)
Requirement already satisfied: httpcore<0.17.0,>=0.15.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from httpx->gen3==4.15.0->-r requirements.txt (line 3)) (0.16.3)
Requirement already satisfied: pytz>=2020.1 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pandas<2.0.0,>=1.4.2->gen3==4.15.0->-r requirements.txt (line 3)) (2022.7.1)
Requirement already satisfied: numpy>=1.20.3 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pandas<2.0.0,>=1.4.2->gen3==4.15.0->-r requirements.txt (line 3)) (1.24.1)
Requirement already satisfied: wcwidth in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from PrettyTable>=0.7.2->python-irodsclient==1.1.4->-r requirements.txt (line 6)) (0.2.6)
Requirement already satisfied: importlib_metadata<2.0.0,>=1.3.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pypfb<1.0.0->gen3==4.15.0->-r requirements.txt (line 3)) (1.7.0)
Requirement already satisfied: fastavro<=1.4.1 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pypfb<1.0.0->gen3==4.15.0->-r requirements.txt (line 3)) (1.4.1)
Requirement already satisfied: PyYAML<6.0.0,>=5.3.1 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pypfb<1.0.0->gen3==4.15.0->-r requirements.txt (line 3)) (5.4.1)
Requirement already satisfied: python-json-logger<0.2.0,>=0.1.11 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pypfb<1.0.0->gen3==4.15.0->-r requirements.txt (line 3)) (0.1.11)
Requirement already satisfied: dictionaryutils<4.0.0,>=3.4.3 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pypfb<1.0.0->gen3==4.15.0->-r requirements.txt (line 3)) (3.4.4)
Requirement already satisfied: gdcdictionary<2.0.0,>=1.2.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pypfb<1.0.0->gen3==4.15.0->-r requirements.txt (line 3)) (1.2.0)
Requirement already satisfied: yarl<2.0,>=1.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from aiohttp->gen3==4.15.0->-r requirements.txt (line 3)) (1.8.2)
Requirement already satisfied: attrs>=17.3.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from aiohttp->gen3==4.15.0->-r requirements.txt (line 3)) (22.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from aiohttp->gen3==4.15.0->-r requirements.txt (line 3)) (1.3.3)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from aiohttp->gen3==4.15.0->-r requirements.txt (line 3)) (4.0.2)
Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from aiohttp->gen3==4.15.0->-r requirements.txt (line 3)) (2.1.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from aiohttp->gen3==4.15.0->-r requirements.txt (line 3)) (6.0.4)
Requirement already satisfied: aiosignal>=1.1.2 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from aiohttp->gen3==4.15.0->-r requirements.txt (line 3)) (1.3.1)
Requirement already satisfied: idna<4,>=2.5 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from requests->gen3==4.15.0->-r requirements.txt (line 3)) (3.4)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from requests->gen3==4.15.0->-r requirements.txt (line 3)) (1.26.14)
Requirement already satisfied: marshmallow-enum<2.0.0,>=1.5.1 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from dataclasses-json->gen3==4.15.0->-r requirements.txt (line 3)) (1.5.1)
Requirement already satisfied: typing-inspect>=0.4.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from dataclasses-json->gen3==4.15.0->-r requirements.txt (line 3)) (0.8.0)
Requirement already satisfied: marshmallow<4.0.0,>=3.3.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from dataclasses-json->gen3==4.15.0->-r requirements.txt (line 3)) (3.19.0)
Requirement already satisfied: pyrsistent>=0.14.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from jsonschema->gen3==4.15.0->-r requirements.txt (line 3)) (0.19.3)
Requirement already satisfied: zipp>=0.5 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from importlib_metadata<2.0.0,>=1.3.0->pypfb<1.0.0->gen3==4.15.0->-r requirements.txt (line 3)) (3.11.0)
Requirement already satisfied: packaging>=17.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from marshmallow<4.0.0,>=3.3.0->dataclasses-json->gen3==4.15.0->-r requirements.txt (line 3)) (23.0)
Requirement already satisfied: mypy-extensions>=0.3.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from typing-inspect>=0.4.0->dataclasses-json->gen3==4.15.0->-r requirements.txt (line 3)) (0.4.3)
[notice] A new release of pip available: 22.3.1 -> 23.1.1
[notice] To update, run: pip install --upgrade pip
+ pip install -r requirements-dev.txt
Requirement already satisfied: pytest==7.1.2 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from -r requirements-dev.txt (line 1)) (7.1.2)
Requirement already satisfied: iniconfig in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pytest==7.1.2->-r requirements-dev.txt (line 1)) (2.0.0)
Requirement already satisfied: pluggy<2.0,>=0.12 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pytest==7.1.2->-r requirements-dev.txt (line 1)) (1.0.0)
Requirement already satisfied: tomli>=1.0.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pytest==7.1.2->-r requirements-dev.txt (line 1)) (2.0.1)
Requirement already satisfied: attrs>=19.2.0 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pytest==7.1.2->-r requirements-dev.txt (line 1)) (22.2.0)
Requirement already satisfied: py>=1.8.2 in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pytest==7.1.2->-r requirements-dev.txt (line 1)) (1.11.0)
Requirement already satisfied: packaging in /home/cmiss/Jenkins/shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages (from pytest==7.1.2->-r requirements-dev.txt (line 1)) (23.0)
[notice] A new release of pip available: 22.3.1 -> 23.1.1
[notice] To update, run: pip install --upgrade pip
+ pytest
============================= test session starts ==============================
platform linux -- Python 3.9.16, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/cmiss/Jenkins/workspace/12-Labours-API
plugins: anyio-3.6.2
collected 17 items
tests/test_api.py FFFFFFF.F... [ 70%]
tests/test_data.py FFFF. [100%]
=================================== FAILURES ===================================
____________________________ test_get_gen3_program _____________________________
client = <starlette.testclient.TestClient object at 0x7f8b76cbd580>
def test_get_gen3_program(client):
response = client.get("/program")
result = response.json()
> assert response.status_code == 200
E assert 404 == 200
E + where 404 = <Response [404]>.status_code
tests/test_api.py:15: AssertionError
____________________________ test_get_gen3_project _____________________________
client = <starlette.testclient.TestClient object at 0x7f8b76c43d00>
def test_get_gen3_project(client):
response = client.get("/project/demo1")
result = response.json()
> assert response.status_code == 200
E assert 404 == 200
E + where 404 = <Response [404]>.status_code
tests/test_api.py:23: AssertionError
___________________________ test_get_gen3_dictionary ___________________________
client = <starlette.testclient.TestClient object at 0x7f8b76b83c70>
def test_get_gen3_dictionary(client):
pass_case = {
"program": "demo1",
"project": "12L",
}
response = client.post("/dictionary", json=pass_case)
> assert response.status_code == 200
E assert 404 == 200
E + where 404 = <Response [404]>.status_code
tests/test_api.py:37: AssertionError
__________________________ test_get_gen3_node_records __________________________
self = <urllib3.connection.HTTPSConnection object at 0x7f8b76b8ddf0>
def _new_conn(self):
"""Establish a socket connection and set nodelay settings on it.
:return: New socket connection.
"""
extra_kw = {}
if self.source_address:
extra_kw["source_address"] = self.source_address
if self.socket_options:
extra_kw["socket_options"] = self.socket_options
try:
> conn = connection.create_connection(
(self._dns_host, self.port), self.timeout, **extra_kw
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:174:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
address = ('gen3.abi-ctt-ctp.cloud.edu.au', 443), timeout = None
source_address = None, socket_options = [(6, 1, 1)]
def create_connection(
address,
timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
source_address=None,
socket_options=None,
):
"""Connect to *address* and return the socket object.
Convenience function. Connect to *address* (a 2-tuple ``(host,
port)``) and return the socket object. Passing the optional
*timeout* parameter will set the timeout on the socket instance
before attempting to connect. If no *timeout* is supplied, the
global default timeout setting returned by :func:`socket.getdefaulttimeout`
is used. If *source_address* is set it must be a tuple of (host, port)
for the socket to bind as a source address before making the connection.
An host of '' or port 0 tells the OS to use the default.
"""
host, port = address
if host.startswith("["):
host = host.strip("[]")
err = None
# Using the value from allowed_gai_family() in the context of getaddrinfo lets
# us select whether to work with IPv4 DNS records, IPv6 records, or both.
# The original create_connection function always returns all records.
family = allowed_gai_family()
try:
host.encode("idna")
except UnicodeError:
return six.raise_from(
LocationParseError(u"'%s', label empty or too long" % host), None
)
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
af, socktype, proto, canonname, sa = res
sock = None
try:
sock = socket.socket(af, socktype, proto)
# If provided, set socket level options before connecting.
_set_socket_options(sock, socket_options)
if timeout is not socket._GLOBAL_DEFAULT_TIMEOUT:
sock.settimeout(timeout)
if source_address:
sock.bind(source_address)
sock.connect(sa)
return sock
except socket.error as e:
err = e
if sock is not None:
sock.close()
sock = None
if err is not None:
> raise err
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/connection.py:95:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
address = ('gen3.abi-ctt-ctp.cloud.edu.au', 443), timeout = None
source_address = None, socket_options = [(6, 1, 1)]
def create_connection(
address,
timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
source_address=None,
socket_options=None,
):
"""Connect to *address* and return the socket object.
Convenience function. Connect to *address* (a 2-tuple ``(host,
port)``) and return the socket object. Passing the optional
*timeout* parameter will set the timeout on the socket instance
before attempting to connect. If no *timeout* is supplied, the
global default timeout setting returned by :func:`socket.getdefaulttimeout`
is used. If *source_address* is set it must be a tuple of (host, port)
for the socket to bind as a source address before making the connection.
An host of '' or port 0 tells the OS to use the default.
"""
host, port = address
if host.startswith("["):
host = host.strip("[]")
err = None
# Using the value from allowed_gai_family() in the context of getaddrinfo lets
# us select whether to work with IPv4 DNS records, IPv6 records, or both.
# The original create_connection function always returns all records.
family = allowed_gai_family()
try:
host.encode("idna")
except UnicodeError:
return six.raise_from(
LocationParseError(u"'%s', label empty or too long" % host), None
)
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
af, socktype, proto, canonname, sa = res
sock = None
try:
sock = socket.socket(af, socktype, proto)
# If provided, set socket level options before connecting.
_set_socket_options(sock, socket_options)
if timeout is not socket._GLOBAL_DEFAULT_TIMEOUT:
sock.settimeout(timeout)
if source_address:
sock.bind(source_address)
> sock.connect(sa)
E TimeoutError: [Errno 110] Connection timed out
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/connection.py:85: TimeoutError
During handling of the above exception, another exception occurred:
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b76a479d0>
method = 'POST', url = '/user/credentials/cdis/access_token'
body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...vPrcq8DT5FB92JLhLStE4nkB9Ysm96h77xFJ44rGq3TODjZaY6Tdj1JgHQrf8BoV2Q", "key_id": "cc7139d4-a9a2-45db-98f8-309087b6b674"}'
headers = {'User-Agent': 'python-requests/2.28.2', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}
retries = Retry(total=0, connect=None, read=False, redirect=None, status=None)
redirect = False, assert_same_host = False
timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None
release_conn = False, chunked = False, body_pos = None
response_kw = {'decode_content': False, 'preload_content': False}
parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None)
destination_scheme = None, conn = None, release_this_conn = True
http_tunnel_required = False, err = None, clean_exit = False
def urlopen(
self,
method,
url,
body=None,
headers=None,
retries=None,
redirect=True,
assert_same_host=True,
timeout=_Default,
pool_timeout=None,
release_conn=None,
chunked=False,
body_pos=None,
**response_kw
):
"""
Get a connection from the pool and perform an HTTP request. This is the
lowest level call for making a request, so you'll need to specify all
the raw details.
.. note::
More commonly, it's appropriate to use a convenience method provided
by :class:`.RequestMethods`, such as :meth:`request`.
.. note::
`release_conn` will only behave as expected if
`preload_content=False` because we want to make
`preload_content=False` the default behaviour someday soon without
breaking backwards compatibility.
:param method:
HTTP request method (such as GET, POST, PUT, etc.)
:param url:
The URL to perform the request on.
:param body:
Data to send in the request body, either :class:`str`, :class:`bytes`,
an iterable of :class:`str`/:class:`bytes`, or a file-like object.
:param headers:
Dictionary of custom headers to send, such as User-Agent,
If-None-Match, etc. If None, pool headers are used. If provided,
these headers completely replace any pool-specific headers.
:param retries:
Configure the number of retries to allow before raising a
:class:`~urllib3.exceptions.MaxRetryError` exception.
Pass ``None`` to retry until you receive a response. Pass a
:class:`~urllib3.util.retry.Retry` object for fine-grained control
over different types of retries.
Pass an integer number to retry connection errors that many times,
but no other types of errors. Pass zero to never retry.
If ``False``, then retries are disabled and any exception is raised
immediately. Also, instead of raising a MaxRetryError on redirects,
the redirect response will be returned.
:type retries: :class:`~urllib3.util.retry.Retry`, False, or an int.
:param redirect:
If True, automatically handle redirects (status codes 301, 302,
303, 307, 308). Each redirect counts as a retry. Disabling retries
will disable redirect, too.
:param assert_same_host:
If ``True``, will make sure that the host of the pool requests is
consistent else will raise HostChangedError. When ``False``, you can
use the pool on an HTTP proxy and request foreign hosts.
:param timeout:
If specified, overrides the default timeout for this one
request. It may be a float (in seconds) or an instance of
:class:`urllib3.util.Timeout`.
:param pool_timeout:
If set and the pool is set to block=True, then this method will
block for ``pool_timeout`` seconds and raise EmptyPoolError if no
connection is available within the time period.
:param release_conn:
If False, then the urlopen call will not release the connection
back into the pool once a response is received (but will release if
you read the entire contents of the response such as when
`preload_content=True`). This is useful if you're not preloading
the response's content immediately. You will need to call
``r.release_conn()`` on the response ``r`` to return the connection
back into the pool. If None, it takes the value of
``response_kw.get('preload_content', True)``.
:param chunked:
If True, urllib3 will send the body using chunked transfer
encoding. Otherwise, urllib3 will send the body using the standard
content-length form. Defaults to False.
:param int body_pos:
Position to seek to in file-like body in the event of a retry or
redirect. Typically this won't need to be set because urllib3 will
auto-populate the value when needed.
:param \\**response_kw:
Additional parameters are passed to
:meth:`urllib3.response.HTTPResponse.from_httplib`
"""
parsed_url = parse_url(url)
destination_scheme = parsed_url.scheme
if headers is None:
headers = self.headers
if not isinstance(retries, Retry):
retries = Retry.from_int(retries, redirect=redirect, default=self.retries)
if release_conn is None:
release_conn = response_kw.get("preload_content", True)
# Check host
if assert_same_host and not self.is_same_host(url):
raise HostChangedError(self, url, retries)
# Ensure that the URL we're connecting to is properly encoded
if url.startswith("/"):
url = six.ensure_str(_encode_target(url))
else:
url = six.ensure_str(parsed_url.url)
conn = None
# Track whether `conn` needs to be released before
# returning/raising/recursing. Update this variable if necessary, and
# leave `release_conn` constant throughout the function. That way, if
# the function recurses, the original value of `release_conn` will be
# passed down into the recursive call, and its value will be respected.
#
# See issue #651 [1] for details.
#
# [1] <https://github.com/urllib3/urllib3/issues/651>
release_this_conn = release_conn
http_tunnel_required = connection_requires_http_tunnel(
self.proxy, self.proxy_config, destination_scheme
)
# Merge the proxy headers. Only done when not using HTTP CONNECT. We
# have to copy the headers dict so we can safely change it without those
# changes being reflected in anyone else's copy.
if not http_tunnel_required:
headers = headers.copy()
headers.update(self.proxy_headers)
# Must keep the exception bound to a separate variable or else Python 3
# complains about UnboundLocalError.
err = None
# Keep track of whether we cleanly exited the except block. This
# ensures we do proper cleanup in finally.
clean_exit = False
# Rewind body position, if needed. Record current position
# for future rewinds in the event of a redirect/retry.
body_pos = set_file_position(body, body_pos)
try:
# Request a connection from the queue.
timeout_obj = self._get_timeout(timeout)
conn = self._get_conn(timeout=pool_timeout)
conn.timeout = timeout_obj.connect_timeout
is_new_proxy_conn = self.proxy is not None and not getattr(
conn, "sock", None
)
if is_new_proxy_conn and http_tunnel_required:
self._prepare_proxy(conn)
# Make the request on the httplib connection object.
> httplib_response = self._make_request(
conn,
method,
url,
timeout=timeout_obj,
body=body,
headers=headers,
chunked=chunked,
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:703:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b76a479d0>
conn = <urllib3.connection.HTTPSConnection object at 0x7f8b76b8ddf0>
method = 'POST', url = '/user/credentials/cdis/access_token'
timeout = Timeout(connect=None, read=None, total=None), chunked = False
httplib_request_kw = {'body': b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwd...p, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}}
timeout_obj = Timeout(connect=None, read=None, total=None)
def _make_request(
self, conn, method, url, timeout=_Default, chunked=False, **httplib_request_kw
):
"""
Perform a request on a given urllib connection object taken from our
pool.
:param conn:
a connection from one of our connection pools
:param timeout:
Socket timeout in seconds for the request. This can be a
float or integer, which will set the same timeout value for
the socket connect and the socket read, or an instance of
:class:`urllib3.util.Timeout`, which gives you more fine-grained
control over your timeouts.
"""
self.num_requests += 1
timeout_obj = self._get_timeout(timeout)
timeout_obj.start_connect()
conn.timeout = timeout_obj.connect_timeout
# Trigger any extra validation we need to do.
try:
> self._validate_conn(conn)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:386:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b76a479d0>
conn = <urllib3.connection.HTTPSConnection object at 0x7f8b76b8ddf0>
def _validate_conn(self, conn):
"""
Called right before a request is made, after the socket is created.
"""
super(HTTPSConnectionPool, self)._validate_conn(conn)
# Force connect early to allow us to validate the connection.
if not getattr(conn, "sock", None): # AppEngine might not have `.sock`
> conn.connect()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:1042:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connection.HTTPSConnection object at 0x7f8b76b8ddf0>
def connect(self):
# Add certificate verification
> self.sock = conn = self._new_conn()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:358:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connection.HTTPSConnection object at 0x7f8b76b8ddf0>
def _new_conn(self):
"""Establish a socket connection and set nodelay settings on it.
:return: New socket connection.
"""
extra_kw = {}
if self.source_address:
extra_kw["source_address"] = self.source_address
if self.socket_options:
extra_kw["socket_options"] = self.socket_options
try:
conn = connection.create_connection(
(self._dns_host, self.port), self.timeout, **extra_kw
)
except SocketTimeout:
raise ConnectTimeoutError(
self,
"Connection to %s timed out. (connect timeout=%s)"
% (self.host, self.timeout),
)
except SocketError as e:
> raise NewConnectionError(
self, "Failed to establish a new connection: %s" % e
)
E urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f8b76b8ddf0>: Failed to establish a new connection: [Errno 110] Connection timed out
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:186: NewConnectionError
During handling of the above exception, another exception occurred:
self = <requests.adapters.HTTPAdapter object at 0x7f8b76b67e80>
request = <PreparedRequest [POST]>, stream = False
timeout = Timeout(connect=None, read=None, total=None), verify = True
cert = None, proxies = OrderedDict()
def send(
self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None
):
"""Sends PreparedRequest object. Returns Response object.
:param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
:param stream: (optional) Whether to stream the request content.
:param timeout: (optional) How long to wait for the server to send
data before giving up, as a float, or a :ref:`(connect timeout,
read timeout) <timeouts>` tuple.
:type timeout: float or tuple or urllib3 Timeout object
:param verify: (optional) Either a boolean, in which case it controls whether
we verify the server's TLS certificate, or a string, in which case it
must be a path to a CA bundle to use
:param cert: (optional) Any user-provided SSL certificate to be trusted.
:param proxies: (optional) The proxies dictionary to apply to the request.
:rtype: requests.Response
"""
try:
conn = self.get_connection(request.url, proxies)
except LocationValueError as e:
raise InvalidURL(e, request=request)
self.cert_verify(conn, request.url, verify, cert)
url = self.request_url(request, proxies)
self.add_headers(
request,
stream=stream,
timeout=timeout,
verify=verify,
cert=cert,
proxies=proxies,
)
chunked = not (request.body is None or "Content-Length" in request.headers)
if isinstance(timeout, tuple):
try:
connect, read = timeout
timeout = TimeoutSauce(connect=connect, read=read)
except ValueError:
raise ValueError(
f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, "
f"or a single float to set both timeouts to the same value."
)
elif isinstance(timeout, TimeoutSauce):
pass
else:
timeout = TimeoutSauce(connect=timeout, read=timeout)
try:
if not chunked:
> resp = conn.urlopen(
method=request.method,
url=url,
body=request.body,
headers=request.headers,
redirect=False,
assert_same_host=False,
preload_content=False,
decode_content=False,
retries=self.max_retries,
timeout=timeout,
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:489:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b76a479d0>
method = 'POST', url = '/user/credentials/cdis/access_token'
body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...vPrcq8DT5FB92JLhLStE4nkB9Ysm96h77xFJ44rGq3TODjZaY6Tdj1JgHQrf8BoV2Q", "key_id": "cc7139d4-a9a2-45db-98f8-309087b6b674"}'
headers = {'User-Agent': 'python-requests/2.28.2', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}
retries = Retry(total=0, connect=None, read=False, redirect=None, status=None)
redirect = False, assert_same_host = False
timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None
release_conn = False, chunked = False, body_pos = None
response_kw = {'decode_content': False, 'preload_content': False}
parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None)
destination_scheme = None, conn = None, release_this_conn = True
http_tunnel_required = False, err = None, clean_exit = False
def urlopen(
self,
method,
url,
body=None,
headers=None,
retries=None,
redirect=True,
assert_same_host=True,
timeout=_Default,
pool_timeout=None,
release_conn=None,
chunked=False,
body_pos=None,
**response_kw
):
"""
Get a connection from the pool and perform an HTTP request. This is the
lowest level call for making a request, so you'll need to specify all
the raw details.
.. note::
More commonly, it's appropriate to use a convenience method provided
by :class:`.RequestMethods`, such as :meth:`request`.
.. note::
`release_conn` will only behave as expected if
`preload_content=False` because we want to make
`preload_content=False` the default behaviour someday soon without
breaking backwards compatibility.
:param method:
HTTP request method (such as GET, POST, PUT, etc.)
:param url:
The URL to perform the request on.
:param body:
Data to send in the request body, either :class:`str`, :class:`bytes`,
an iterable of :class:`str`/:class:`bytes`, or a file-like object.
:param headers:
Dictionary of custom headers to send, such as User-Agent,
If-None-Match, etc. If None, pool headers are used. If provided,
these headers completely replace any pool-specific headers.
:param retries:
Configure the number of retries to allow before raising a
:class:`~urllib3.exceptions.MaxRetryError` exception.
Pass ``None`` to retry until you receive a response. Pass a
:class:`~urllib3.util.retry.Retry` object for fine-grained control
over different types of retries.
Pass an integer number to retry connection errors that many times,
but no other types of errors. Pass zero to never retry.
If ``False``, then retries are disabled and any exception is raised
immediately. Also, instead of raising a MaxRetryError on redirects,
the redirect response will be returned.
:type retries: :class:`~urllib3.util.retry.Retry`, False, or an int.
:param redirect:
If True, automatically handle redirects (status codes 301, 302,
303, 307, 308). Each redirect counts as a retry. Disabling retries
will disable redirect, too.
:param assert_same_host:
If ``True``, will make sure that the host of the pool requests is
consistent else will raise HostChangedError. When ``False``, you can
use the pool on an HTTP proxy and request foreign hosts.
:param timeout:
If specified, overrides the default timeout for this one
request. It may be a float (in seconds) or an instance of
:class:`urllib3.util.Timeout`.
:param pool_timeout:
If set and the pool is set to block=True, then this method will
block for ``pool_timeout`` seconds and raise EmptyPoolError if no
connection is available within the time period.
:param release_conn:
If False, then the urlopen call will not release the connection
back into the pool once a response is received (but will release if
you read the entire contents of the response such as when
`preload_content=True`). This is useful if you're not preloading
the response's content immediately. You will need to call
``r.release_conn()`` on the response ``r`` to return the connection
back into the pool. If None, it takes the value of
``response_kw.get('preload_content', True)``.
:param chunked:
If True, urllib3 will send the body using chunked transfer
encoding. Otherwise, urllib3 will send the body using the standard
content-length form. Defaults to False.
:param int body_pos:
Position to seek to in file-like body in the event of a retry or
redirect. Typically this won't need to be set because urllib3 will
auto-populate the value when needed.
:param \\**response_kw:
Additional parameters are passed to
:meth:`urllib3.response.HTTPResponse.from_httplib`
"""
parsed_url = parse_url(url)
destination_scheme = parsed_url.scheme
if headers is None:
headers = self.headers
if not isinstance(retries, Retry):
retries = Retry.from_int(retries, redirect=redirect, default=self.retries)
if release_conn is None:
release_conn = response_kw.get("preload_content", True)
# Check host
if assert_same_host and not self.is_same_host(url):
raise HostChangedError(self, url, retries)
# Ensure that the URL we're connecting to is properly encoded
if url.startswith("/"):
url = six.ensure_str(_encode_target(url))
else:
url = six.ensure_str(parsed_url.url)
conn = None
# Track whether `conn` needs to be released before
# returning/raising/recursing. Update this variable if necessary, and
# leave `release_conn` constant throughout the function. That way, if
# the function recurses, the original value of `release_conn` will be
# passed down into the recursive call, and its value will be respected.
#
# See issue #651 [1] for details.
#
# [1] <https://github.com/urllib3/urllib3/issues/651>
release_this_conn = release_conn
http_tunnel_required = connection_requires_http_tunnel(
self.proxy, self.proxy_config, destination_scheme
)
# Merge the proxy headers. Only done when not using HTTP CONNECT. We
# have to copy the headers dict so we can safely change it without those
# changes being reflected in anyone else's copy.
if not http_tunnel_required:
headers = headers.copy()
headers.update(self.proxy_headers)
# Must keep the exception bound to a separate variable or else Python 3
# complains about UnboundLocalError.
err = None
# Keep track of whether we cleanly exited the except block. This
# ensures we do proper cleanup in finally.
clean_exit = False
# Rewind body position, if needed. Record current position
# for future rewinds in the event of a redirect/retry.
body_pos = set_file_position(body, body_pos)
try:
# Request a connection from the queue.
timeout_obj = self._get_timeout(timeout)
conn = self._get_conn(timeout=pool_timeout)
conn.timeout = timeout_obj.connect_timeout
is_new_proxy_conn = self.proxy is not None and not getattr(
conn, "sock", None
)
if is_new_proxy_conn and http_tunnel_required:
self._prepare_proxy(conn)
# Make the request on the httplib connection object.
httplib_response = self._make_request(
conn,
method,
url,
timeout=timeout_obj,
body=body,
headers=headers,
chunked=chunked,
)
# If we're going to release the connection in ``finally:``, then
# the response doesn't need to know about the connection. Otherwise
# it will also try to release it and we'll have a double-release
# mess.
response_conn = conn if not release_conn else None
# Pass method to Response for length checking
response_kw["request_method"] = method
# Import httplib's response into our own wrapper object
response = self.ResponseCls.from_httplib(
httplib_response,
pool=self,
connection=response_conn,
retries=retries,
**response_kw
)
# Everything went great!
clean_exit = True
except EmptyPoolError:
# Didn't get a connection from the pool, no need to clean up
clean_exit = True
release_this_conn = False
raise
except (
TimeoutError,
HTTPException,
SocketError,
ProtocolError,
BaseSSLError,
SSLError,
CertificateError,
) as e:
# Discard the connection for these exceptions. It will be
# replaced during the next _get_conn() call.
clean_exit = False
def _is_ssl_error_message_from_http_proxy(ssl_error):
# We're trying to detect the message 'WRONG_VERSION_NUMBER' but
# SSLErrors are kinda all over the place when it comes to the message,
# so we try to cover our bases here!
message = " ".join(re.split("[^a-z]", str(ssl_error).lower()))
return (
"wrong version number" in message or "unknown protocol" in message
)
# Try to detect a common user error with proxies which is to
# set an HTTP proxy to be HTTPS when it should be 'http://'
# (ie {'http': 'http://proxy', 'https': 'https://proxy'})
# Instead we add a nice error message and point to a URL.
if (
isinstance(e, BaseSSLError)
and self.proxy
and _is_ssl_error_message_from_http_proxy(e)
and conn.proxy
and conn.proxy.scheme == "https"
):
e = ProxyError(
"Your proxy appears to only use HTTP and not HTTPS, "
"try changing your proxy URL to be HTTP. See: "
"https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html"
"#https-proxy-error-http-proxy",
SSLError(e),
)
elif isinstance(e, (BaseSSLError, CertificateError)):
e = SSLError(e)
elif isinstance(e, (SocketError, NewConnectionError)) and self.proxy:
e = ProxyError("Cannot connect to proxy.", e)
elif isinstance(e, (SocketError, HTTPException)):
e = ProtocolError("Connection aborted.", e)
> retries = retries.increment(
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:787:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = Retry(total=0, connect=None, read=False, redirect=None, status=None)
method = 'POST', url = '/user/credentials/cdis/access_token', response = None
error = NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8b76b8ddf0>: Failed to establish a new connection: [Errno 110] Connection timed out')
_pool = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b76a479d0>
_stacktrace = <traceback object at 0x7f8b76be20c0>
def increment(
self,
method=None,
url=None,
response=None,
error=None,
_pool=None,
_stacktrace=None,
):
"""Return a new Retry object with incremented retry counters.
:param response: A response object, or None, if the server did not
return a response.
:type response: :class:`~urllib3.response.HTTPResponse`
:param Exception error: An error encountered during the request, or
None if the response was received successfully.
:return: A new ``Retry`` object.
"""
if self.total is False and error:
# Disabled, indicate to re-raise the error.
raise six.reraise(type(error), error, _stacktrace)
total = self.total
if total is not None:
total -= 1
connect = self.connect
read = self.read
redirect = self.redirect
status_count = self.status
other = self.other
cause = "unknown"
status = None
redirect_location = None
if error and self._is_connection_error(error):
# Connect retry?
if connect is False:
raise six.reraise(type(error), error, _stacktrace)
elif connect is not None:
connect -= 1
elif error and self._is_read_error(error):
# Read retry?
if read is False or not self._is_method_retryable(method):
raise six.reraise(type(error), error, _stacktrace)
elif read is not None:
read -= 1
elif error:
# Other retry?
if other is not None:
other -= 1
elif response and response.get_redirect_location():
# Redirect retry?
if redirect is not None:
redirect -= 1
cause = "too many redirects"
redirect_location = response.get_redirect_location()
status = response.status
else:
# Incrementing because of a server error like a 500 in
# status_forcelist and the given method is in the allowed_methods
cause = ResponseError.GENERIC_ERROR
if response and response.status:
if status_count is not None:
status_count -= 1
cause = ResponseError.SPECIFIC_ERROR.format(status_code=response.status)
status = response.status
history = self.history + (
RequestHistory(method, url, error, status, redirect_location),
)
new_retry = self.new(
total=total,
connect=connect,
read=read,
redirect=redirect,
status=status_count,
other=other,
history=history,
)
if new_retry.is_exhausted():
> raise MaxRetryError(_pool, url, error or ResponseError(cause))
E urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8b76b8ddf0>: Failed to establish a new connection: [Errno 110] Connection timed out'))
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/retry.py:592: MaxRetryError
During handling of the above exception, another exception occurred:
client = <starlette.testclient.TestClient object at 0x7f8b76b833a0>
def test_get_gen3_node_records(client):
NODE_TYPE = "experiment"
pass_case = {
"program": "demo1",
"project": "12L",
}
> response = client.post(f"/records/{NODE_TYPE}", json=pass_case)
tests/test_api.py:62:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:635: in post
return self.request("POST", url, data=data, json=json, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:476: in request
return super().request(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:587: in request
resp = self.send(prep, **send_kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:701: in send
r = adapter.send(request, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:270: in send
raise exc
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:267: in send
portal.call(self.app, scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:283: in call
return cast(T_Retval, self.start_task_soon(func, *args).result())
/usr/lib/python3.9/concurrent/futures/_base.py:446: in result
return self.__get_result()
/usr/lib/python3.9/concurrent/futures/_base.py:391: in __get_result
raise self._exception
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:219: in _call_func
retval = await retval
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/applications.py:269: in __call__
await super().__call__(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/applications.py:124: in __call__
await self.middleware_stack(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:184: in __call__
raise exc
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:162: in __call__
await self.app(scope, receive, _send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/cors.py:84: in __call__
await self.app(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:93: in __call__
raise exc
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:82: in __call__
await self.app(scope, receive, sender)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:21: in __call__
raise e
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:18: in __call__
await self.app(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:670: in __call__
await route.handle(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:266: in handle
await self.app(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:65: in app
response = await func(request)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:227: in app
raw_response = await run_endpoint_function(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:160: in run_endpoint_function
return await dependant.call(**values)
app/main.py:192: in get_gen3_node_records
node_record = SUBMISSION.export_node(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/submission.py:384: in export_node
output = requests.get(api_url, auth=self._auth_provider).text
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:73: in get
return request("get", url, params=params, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request
return session.request(method=method, url=url, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:573: in request
prep = self.prepare_request(req)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:484: in prepare_request
p.prepare(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:372: in prepare
self.prepare_auth(auth, url)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:603: in prepare_auth
r = auth(self)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:266: in __call__
request.headers["Authorization"] = self._get_auth_value()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:378: in _get_auth_value
return "bearer " + self.get_access_token()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:367: in get_access_token
return self.refresh_access_token(endpoint)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:304: in refresh_access_token
self._access_token = get_access_token_with_key(self._refresh_token)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:75: in get_access_token_with_key
resp = requests.post(auth_url, json=api_key)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:115: in post
return request("post", url, data=data, json=json, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request
return session.request(method=method, url=url, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:587: in request
resp = self.send(prep, **send_kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:701: in send
r = adapter.send(request, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <requests.adapters.HTTPAdapter object at 0x7f8b76b67e80>
request = <PreparedRequest [POST]>, stream = False
timeout = Timeout(connect=None, read=None, total=None), verify = True
cert = None, proxies = OrderedDict()
def send(
self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None
):
"""Sends PreparedRequest object. Returns Response object.
:param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
:param stream: (optional) Whether to stream the request content.
:param timeout: (optional) How long to wait for the server to send
data before giving up, as a float, or a :ref:`(connect timeout,
read timeout) <timeouts>` tuple.
:type timeout: float or tuple or urllib3 Timeout object
:param verify: (optional) Either a boolean, in which case it controls whether
we verify the server's TLS certificate, or a string, in which case it
must be a path to a CA bundle to use
:param cert: (optional) Any user-provided SSL certificate to be trusted.
:param proxies: (optional) The proxies dictionary to apply to the request.
:rtype: requests.Response
"""
try:
conn = self.get_connection(request.url, proxies)
except LocationValueError as e:
raise InvalidURL(e, request=request)
self.cert_verify(conn, request.url, verify, cert)
url = self.request_url(request, proxies)
self.add_headers(
request,
stream=stream,
timeout=timeout,
verify=verify,
cert=cert,
proxies=proxies,
)
chunked = not (request.body is None or "Content-Length" in request.headers)
if isinstance(timeout, tuple):
try:
connect, read = timeout
timeout = TimeoutSauce(connect=connect, read=read)
except ValueError:
raise ValueError(
f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, "
f"or a single float to set both timeouts to the same value."
)
elif isinstance(timeout, TimeoutSauce):
pass
else:
timeout = TimeoutSauce(connect=timeout, read=timeout)
try:
if not chunked:
resp = conn.urlopen(
method=request.method,
url=url,
body=request.body,
headers=request.headers,
redirect=False,
assert_same_host=False,
preload_content=False,
decode_content=False,
retries=self.max_retries,
timeout=timeout,
)
# Send the request.
else:
if hasattr(conn, "proxy_pool"):
conn = conn.proxy_pool
low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)
try:
skip_host = "Host" in request.headers
low_conn.putrequest(
request.method,
url,
skip_accept_encoding=True,
skip_host=skip_host,
)
for header, value in request.headers.items():
low_conn.putheader(header, value)
low_conn.endheaders()
for i in request.body:
low_conn.send(hex(len(i))[2:].encode("utf-8"))
low_conn.send(b"\r\n")
low_conn.send(i)
low_conn.send(b"\r\n")
low_conn.send(b"0\r\n\r\n")
# Receive the response from the server
r = low_conn.getresponse()
resp = HTTPResponse.from_httplib(
r,
pool=conn,
connection=low_conn,
preload_content=False,
decode_content=False,
)
except Exception:
# If we hit any problems here, clean up the connection.
# Then, raise so that we can handle the actual exception.
low_conn.close()
raise
except (ProtocolError, OSError) as err:
raise ConnectionError(err, request=request)
except MaxRetryError as e:
if isinstance(e.reason, ConnectTimeoutError):
# TODO: Remove this in 3.0.0: see #2811
if not isinstance(e.reason, NewConnectionError):
raise ConnectTimeout(e, request=request)
if isinstance(e.reason, ResponseError):
raise RetryError(e, request=request)
if isinstance(e.reason, _ProxyError):
raise ProxyError(e, request=request)
if isinstance(e.reason, _SSLError):
# This branch is for urllib3 v1.22 and later.
raise SSLError(e, request=request)
> raise ConnectionError(e, request=request)
E requests.exceptions.ConnectionError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8b76b8ddf0>: Failed to establish a new connection: [Errno 110] Connection timed out'))
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:565: ConnectionError
_____________________________ test_get_gen3_record _____________________________
self = <urllib3.connection.HTTPSConnection object at 0x7f8b75bd0a00>
def _new_conn(self):
"""Establish a socket connection and set nodelay settings on it.
:return: New socket connection.
"""
extra_kw = {}
if self.source_address:
extra_kw["source_address"] = self.source_address
if self.socket_options:
extra_kw["socket_options"] = self.socket_options
try:
> conn = connection.create_connection(
(self._dns_host, self.port), self.timeout, **extra_kw
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:174:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
address = ('gen3.abi-ctt-ctp.cloud.edu.au', 443), timeout = None
source_address = None, socket_options = [(6, 1, 1)]
def create_connection(
address,
timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
source_address=None,
socket_options=None,
):
"""Connect to *address* and return the socket object.
Convenience function. Connect to *address* (a 2-tuple ``(host,
port)``) and return the socket object. Passing the optional
*timeout* parameter will set the timeout on the socket instance
before attempting to connect. If no *timeout* is supplied, the
global default timeout setting returned by :func:`socket.getdefaulttimeout`
is used. If *source_address* is set it must be a tuple of (host, port)
for the socket to bind as a source address before making the connection.
An host of '' or port 0 tells the OS to use the default.
"""
host, port = address
if host.startswith("["):
host = host.strip("[]")
err = None
# Using the value from allowed_gai_family() in the context of getaddrinfo lets
# us select whether to work with IPv4 DNS records, IPv6 records, or both.
# The original create_connection function always returns all records.
family = allowed_gai_family()
try:
host.encode("idna")
except UnicodeError:
return six.raise_from(
LocationParseError(u"'%s', label empty or too long" % host), None
)
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
af, socktype, proto, canonname, sa = res
sock = None
try:
sock = socket.socket(af, socktype, proto)
# If provided, set socket level options before connecting.
_set_socket_options(sock, socket_options)
if timeout is not socket._GLOBAL_DEFAULT_TIMEOUT:
sock.settimeout(timeout)
if source_address:
sock.bind(source_address)
sock.connect(sa)
return sock
except socket.error as e:
err = e
if sock is not None:
sock.close()
sock = None
if err is not None:
> raise err
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/connection.py:95:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
address = ('gen3.abi-ctt-ctp.cloud.edu.au', 443), timeout = None
source_address = None, socket_options = [(6, 1, 1)]
def create_connection(
address,
timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
source_address=None,
socket_options=None,
):
"""Connect to *address* and return the socket object.
Convenience function. Connect to *address* (a 2-tuple ``(host,
port)``) and return the socket object. Passing the optional
*timeout* parameter will set the timeout on the socket instance
before attempting to connect. If no *timeout* is supplied, the
global default timeout setting returned by :func:`socket.getdefaulttimeout`
is used. If *source_address* is set it must be a tuple of (host, port)
for the socket to bind as a source address before making the connection.
An host of '' or port 0 tells the OS to use the default.
"""
host, port = address
if host.startswith("["):
host = host.strip("[]")
err = None
# Using the value from allowed_gai_family() in the context of getaddrinfo lets
# us select whether to work with IPv4 DNS records, IPv6 records, or both.
# The original create_connection function always returns all records.
family = allowed_gai_family()
try:
host.encode("idna")
except UnicodeError:
return six.raise_from(
LocationParseError(u"'%s', label empty or too long" % host), None
)
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
af, socktype, proto, canonname, sa = res
sock = None
try:
sock = socket.socket(af, socktype, proto)
# If provided, set socket level options before connecting.
_set_socket_options(sock, socket_options)
if timeout is not socket._GLOBAL_DEFAULT_TIMEOUT:
sock.settimeout(timeout)
if source_address:
sock.bind(source_address)
> sock.connect(sa)
E TimeoutError: [Errno 110] Connection timed out
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/connection.py:85: TimeoutError
During handling of the above exception, another exception occurred:
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b75b450a0>
method = 'POST', url = '/user/credentials/cdis/access_token'
body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...vPrcq8DT5FB92JLhLStE4nkB9Ysm96h77xFJ44rGq3TODjZaY6Tdj1JgHQrf8BoV2Q", "key_id": "cc7139d4-a9a2-45db-98f8-309087b6b674"}'
headers = {'User-Agent': 'python-requests/2.28.2', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}
retries = Retry(total=0, connect=None, read=False, redirect=None, status=None)
redirect = False, assert_same_host = False
timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None
release_conn = False, chunked = False, body_pos = None
response_kw = {'decode_content': False, 'preload_content': False}
parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None)
destination_scheme = None, conn = None, release_this_conn = True
http_tunnel_required = False, err = None, clean_exit = False
def urlopen(
self,
method,
url,
body=None,
headers=None,
retries=None,
redirect=True,
assert_same_host=True,
timeout=_Default,
pool_timeout=None,
release_conn=None,
chunked=False,
body_pos=None,
**response_kw
):
"""
Get a connection from the pool and perform an HTTP request. This is the
lowest level call for making a request, so you'll need to specify all
the raw details.
.. note::
More commonly, it's appropriate to use a convenience method provided
by :class:`.RequestMethods`, such as :meth:`request`.
.. note::
`release_conn` will only behave as expected if
`preload_content=False` because we want to make
`preload_content=False` the default behaviour someday soon without
breaking backwards compatibility.
:param method:
HTTP request method (such as GET, POST, PUT, etc.)
:param url:
The URL to perform the request on.
:param body:
Data to send in the request body, either :class:`str`, :class:`bytes`,
an iterable of :class:`str`/:class:`bytes`, or a file-like object.
:param headers:
Dictionary of custom headers to send, such as User-Agent,
If-None-Match, etc. If None, pool headers are used. If provided,
these headers completely replace any pool-specific headers.
:param retries:
Configure the number of retries to allow before raising a
:class:`~urllib3.exceptions.MaxRetryError` exception.
Pass ``None`` to retry until you receive a response. Pass a
:class:`~urllib3.util.retry.Retry` object for fine-grained control
over different types of retries.
Pass an integer number to retry connection errors that many times,
but no other types of errors. Pass zero to never retry.
If ``False``, then retries are disabled and any exception is raised
immediately. Also, instead of raising a MaxRetryError on redirects,
the redirect response will be returned.
:type retries: :class:`~urllib3.util.retry.Retry`, False, or an int.
:param redirect:
If True, automatically handle redirects (status codes 301, 302,
303, 307, 308). Each redirect counts as a retry. Disabling retries
will disable redirect, too.
:param assert_same_host:
If ``True``, will make sure that the host of the pool requests is
consistent else will raise HostChangedError. When ``False``, you can
use the pool on an HTTP proxy and request foreign hosts.
:param timeout:
If specified, overrides the default timeout for this one
request. It may be a float (in seconds) or an instance of
:class:`urllib3.util.Timeout`.
:param pool_timeout:
If set and the pool is set to block=True, then this method will
block for ``pool_timeout`` seconds and raise EmptyPoolError if no
connection is available within the time period.
:param release_conn:
If False, then the urlopen call will not release the connection
back into the pool once a response is received (but will release if
you read the entire contents of the response such as when
`preload_content=True`). This is useful if you're not preloading
the response's content immediately. You will need to call
``r.release_conn()`` on the response ``r`` to return the connection
back into the pool. If None, it takes the value of
``response_kw.get('preload_content', True)``.
:param chunked:
If True, urllib3 will send the body using chunked transfer
encoding. Otherwise, urllib3 will send the body using the standard
content-length form. Defaults to False.
:param int body_pos:
Position to seek to in file-like body in the event of a retry or
redirect. Typically this won't need to be set because urllib3 will
auto-populate the value when needed.
:param \\**response_kw:
Additional parameters are passed to
:meth:`urllib3.response.HTTPResponse.from_httplib`
"""
parsed_url = parse_url(url)
destination_scheme = parsed_url.scheme
if headers is None:
headers = self.headers
if not isinstance(retries, Retry):
retries = Retry.from_int(retries, redirect=redirect, default=self.retries)
if release_conn is None:
release_conn = response_kw.get("preload_content", True)
# Check host
if assert_same_host and not self.is_same_host(url):
raise HostChangedError(self, url, retries)
# Ensure that the URL we're connecting to is properly encoded
if url.startswith("/"):
url = six.ensure_str(_encode_target(url))
else:
url = six.ensure_str(parsed_url.url)
conn = None
# Track whether `conn` needs to be released before
# returning/raising/recursing. Update this variable if necessary, and
# leave `release_conn` constant throughout the function. That way, if
# the function recurses, the original value of `release_conn` will be
# passed down into the recursive call, and its value will be respected.
#
# See issue #651 [1] for details.
#
# [1] <https://github.com/urllib3/urllib3/issues/651>
release_this_conn = release_conn
http_tunnel_required = connection_requires_http_tunnel(
self.proxy, self.proxy_config, destination_scheme
)
# Merge the proxy headers. Only done when not using HTTP CONNECT. We
# have to copy the headers dict so we can safely change it without those
# changes being reflected in anyone else's copy.
if not http_tunnel_required:
headers = headers.copy()
headers.update(self.proxy_headers)
# Must keep the exception bound to a separate variable or else Python 3
# complains about UnboundLocalError.
err = None
# Keep track of whether we cleanly exited the except block. This
# ensures we do proper cleanup in finally.
clean_exit = False
# Rewind body position, if needed. Record current position
# for future rewinds in the event of a redirect/retry.
body_pos = set_file_position(body, body_pos)
try:
# Request a connection from the queue.
timeout_obj = self._get_timeout(timeout)
conn = self._get_conn(timeout=pool_timeout)
conn.timeout = timeout_obj.connect_timeout
is_new_proxy_conn = self.proxy is not None and not getattr(
conn, "sock", None
)
if is_new_proxy_conn and http_tunnel_required:
self._prepare_proxy(conn)
# Make the request on the httplib connection object.
> httplib_response = self._make_request(
conn,
method,
url,
timeout=timeout_obj,
body=body,
headers=headers,
chunked=chunked,
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:703:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b75b450a0>
conn = <urllib3.connection.HTTPSConnection object at 0x7f8b75bd0a00>
method = 'POST', url = '/user/credentials/cdis/access_token'
timeout = Timeout(connect=None, read=None, total=None), chunked = False
httplib_request_kw = {'body': b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwd...p, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}}
timeout_obj = Timeout(connect=None, read=None, total=None)
def _make_request(
self, conn, method, url, timeout=_Default, chunked=False, **httplib_request_kw
):
"""
Perform a request on a given urllib connection object taken from our
pool.
:param conn:
a connection from one of our connection pools
:param timeout:
Socket timeout in seconds for the request. This can be a
float or integer, which will set the same timeout value for
the socket connect and the socket read, or an instance of
:class:`urllib3.util.Timeout`, which gives you more fine-grained
control over your timeouts.
"""
self.num_requests += 1
timeout_obj = self._get_timeout(timeout)
timeout_obj.start_connect()
conn.timeout = timeout_obj.connect_timeout
# Trigger any extra validation we need to do.
try:
> self._validate_conn(conn)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:386:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b75b450a0>
conn = <urllib3.connection.HTTPSConnection object at 0x7f8b75bd0a00>
def _validate_conn(self, conn):
"""
Called right before a request is made, after the socket is created.
"""
super(HTTPSConnectionPool, self)._validate_conn(conn)
# Force connect early to allow us to validate the connection.
if not getattr(conn, "sock", None): # AppEngine might not have `.sock`
> conn.connect()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:1042:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connection.HTTPSConnection object at 0x7f8b75bd0a00>
def connect(self):
# Add certificate verification
> self.sock = conn = self._new_conn()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:358:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connection.HTTPSConnection object at 0x7f8b75bd0a00>
def _new_conn(self):
"""Establish a socket connection and set nodelay settings on it.
:return: New socket connection.
"""
extra_kw = {}
if self.source_address:
extra_kw["source_address"] = self.source_address
if self.socket_options:
extra_kw["socket_options"] = self.socket_options
try:
conn = connection.create_connection(
(self._dns_host, self.port), self.timeout, **extra_kw
)
except SocketTimeout:
raise ConnectTimeoutError(
self,
"Connection to %s timed out. (connect timeout=%s)"
% (self.host, self.timeout),
)
except SocketError as e:
> raise NewConnectionError(
self, "Failed to establish a new connection: %s" % e
)
E urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f8b75bd0a00>: Failed to establish a new connection: [Errno 110] Connection timed out
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:186: NewConnectionError
During handling of the above exception, another exception occurred:
self = <requests.adapters.HTTPAdapter object at 0x7f8b75b45e20>
request = <PreparedRequest [POST]>, stream = False
timeout = Timeout(connect=None, read=None, total=None), verify = True
cert = None, proxies = OrderedDict()
def send(
self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None
):
"""Sends PreparedRequest object. Returns Response object.
:param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
:param stream: (optional) Whether to stream the request content.
:param timeout: (optional) How long to wait for the server to send
data before giving up, as a float, or a :ref:`(connect timeout,
read timeout) <timeouts>` tuple.
:type timeout: float or tuple or urllib3 Timeout object
:param verify: (optional) Either a boolean, in which case it controls whether
we verify the server's TLS certificate, or a string, in which case it
must be a path to a CA bundle to use
:param cert: (optional) Any user-provided SSL certificate to be trusted.
:param proxies: (optional) The proxies dictionary to apply to the request.
:rtype: requests.Response
"""
try:
conn = self.get_connection(request.url, proxies)
except LocationValueError as e:
raise InvalidURL(e, request=request)
self.cert_verify(conn, request.url, verify, cert)
url = self.request_url(request, proxies)
self.add_headers(
request,
stream=stream,
timeout=timeout,
verify=verify,
cert=cert,
proxies=proxies,
)
chunked = not (request.body is None or "Content-Length" in request.headers)
if isinstance(timeout, tuple):
try:
connect, read = timeout
timeout = TimeoutSauce(connect=connect, read=read)
except ValueError:
raise ValueError(
f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, "
f"or a single float to set both timeouts to the same value."
)
elif isinstance(timeout, TimeoutSauce):
pass
else:
timeout = TimeoutSauce(connect=timeout, read=timeout)
try:
if not chunked:
> resp = conn.urlopen(
method=request.method,
url=url,
body=request.body,
headers=request.headers,
redirect=False,
assert_same_host=False,
preload_content=False,
decode_content=False,
retries=self.max_retries,
timeout=timeout,
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:489:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b75b450a0>
method = 'POST', url = '/user/credentials/cdis/access_token'
body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...vPrcq8DT5FB92JLhLStE4nkB9Ysm96h77xFJ44rGq3TODjZaY6Tdj1JgHQrf8BoV2Q", "key_id": "cc7139d4-a9a2-45db-98f8-309087b6b674"}'
headers = {'User-Agent': 'python-requests/2.28.2', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}
retries = Retry(total=0, connect=None, read=False, redirect=None, status=None)
redirect = False, assert_same_host = False
timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None
release_conn = False, chunked = False, body_pos = None
response_kw = {'decode_content': False, 'preload_content': False}
parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None)
destination_scheme = None, conn = None, release_this_conn = True
http_tunnel_required = False, err = None, clean_exit = False
def urlopen(
self,
method,
url,
body=None,
headers=None,
retries=None,
redirect=True,
assert_same_host=True,
timeout=_Default,
pool_timeout=None,
release_conn=None,
chunked=False,
body_pos=None,
**response_kw
):
"""
Get a connection from the pool and perform an HTTP request. This is the
lowest level call for making a request, so you'll need to specify all
the raw details.
.. note::
More commonly, it's appropriate to use a convenience method provided
by :class:`.RequestMethods`, such as :meth:`request`.
.. note::
`release_conn` will only behave as expected if
`preload_content=False` because we want to make
`preload_content=False` the default behaviour someday soon without
breaking backwards compatibility.
:param method:
HTTP request method (such as GET, POST, PUT, etc.)
:param url:
The URL to perform the request on.
:param body:
Data to send in the request body, either :class:`str`, :class:`bytes`,
an iterable of :class:`str`/:class:`bytes`, or a file-like object.
:param headers:
Dictionary of custom headers to send, such as User-Agent,
If-None-Match, etc. If None, pool headers are used. If provided,
these headers completely replace any pool-specific headers.
:param retries:
Configure the number of retries to allow before raising a
:class:`~urllib3.exceptions.MaxRetryError` exception.
Pass ``None`` to retry until you receive a response. Pass a
:class:`~urllib3.util.retry.Retry` object for fine-grained control
over different types of retries.
Pass an integer number to retry connection errors that many times,
but no other types of errors. Pass zero to never retry.
If ``False``, then retries are disabled and any exception is raised
immediately. Also, instead of raising a MaxRetryError on redirects,
the redirect response will be returned.
:type retries: :class:`~urllib3.util.retry.Retry`, False, or an int.
:param redirect:
If True, automatically handle redirects (status codes 301, 302,
303, 307, 308). Each redirect counts as a retry. Disabling retries
will disable redirect, too.
:param assert_same_host:
If ``True``, will make sure that the host of the pool requests is
consistent else will raise HostChangedError. When ``False``, you can
use the pool on an HTTP proxy and request foreign hosts.
:param timeout:
If specified, overrides the default timeout for this one
request. It may be a float (in seconds) or an instance of
:class:`urllib3.util.Timeout`.
:param pool_timeout:
If set and the pool is set to block=True, then this method will
block for ``pool_timeout`` seconds and raise EmptyPoolError if no
connection is available within the time period.
:param release_conn:
If False, then the urlopen call will not release the connection
back into the pool once a response is received (but will release if
you read the entire contents of the response such as when
`preload_content=True`). This is useful if you're not preloading
the response's content immediately. You will need to call
``r.release_conn()`` on the response ``r`` to return the connection
back into the pool. If None, it takes the value of
``response_kw.get('preload_content', True)``.
:param chunked:
If True, urllib3 will send the body using chunked transfer
encoding. Otherwise, urllib3 will send the body using the standard
content-length form. Defaults to False.
:param int body_pos:
Position to seek to in file-like body in the event of a retry or
redirect. Typically this won't need to be set because urllib3 will
auto-populate the value when needed.
:param \\**response_kw:
Additional parameters are passed to
:meth:`urllib3.response.HTTPResponse.from_httplib`
"""
parsed_url = parse_url(url)
destination_scheme = parsed_url.scheme
if headers is None:
headers = self.headers
if not isinstance(retries, Retry):
retries = Retry.from_int(retries, redirect=redirect, default=self.retries)
if release_conn is None:
release_conn = response_kw.get("preload_content", True)
# Check host
if assert_same_host and not self.is_same_host(url):
raise HostChangedError(self, url, retries)
# Ensure that the URL we're connecting to is properly encoded
if url.startswith("/"):
url = six.ensure_str(_encode_target(url))
else:
url = six.ensure_str(parsed_url.url)
conn = None
# Track whether `conn` needs to be released before
# returning/raising/recursing. Update this variable if necessary, and
# leave `release_conn` constant throughout the function. That way, if
# the function recurses, the original value of `release_conn` will be
# passed down into the recursive call, and its value will be respected.
#
# See issue #651 [1] for details.
#
# [1] <https://github.com/urllib3/urllib3/issues/651>
release_this_conn = release_conn
http_tunnel_required = connection_requires_http_tunnel(
self.proxy, self.proxy_config, destination_scheme
)
# Merge the proxy headers. Only done when not using HTTP CONNECT. We
# have to copy the headers dict so we can safely change it without those
# changes being reflected in anyone else's copy.
if not http_tunnel_required:
headers = headers.copy()
headers.update(self.proxy_headers)
# Must keep the exception bound to a separate variable or else Python 3
# complains about UnboundLocalError.
err = None
# Keep track of whether we cleanly exited the except block. This
# ensures we do proper cleanup in finally.
clean_exit = False
# Rewind body position, if needed. Record current position
# for future rewinds in the event of a redirect/retry.
body_pos = set_file_position(body, body_pos)
try:
# Request a connection from the queue.
timeout_obj = self._get_timeout(timeout)
conn = self._get_conn(timeout=pool_timeout)
conn.timeout = timeout_obj.connect_timeout
is_new_proxy_conn = self.proxy is not None and not getattr(
conn, "sock", None
)
if is_new_proxy_conn and http_tunnel_required:
self._prepare_proxy(conn)
# Make the request on the httplib connection object.
httplib_response = self._make_request(
conn,
method,
url,
timeout=timeout_obj,
body=body,
headers=headers,
chunked=chunked,
)
# If we're going to release the connection in ``finally:``, then
# the response doesn't need to know about the connection. Otherwise
# it will also try to release it and we'll have a double-release
# mess.
response_conn = conn if not release_conn else None
# Pass method to Response for length checking
response_kw["request_method"] = method
# Import httplib's response into our own wrapper object
response = self.ResponseCls.from_httplib(
httplib_response,
pool=self,
connection=response_conn,
retries=retries,
**response_kw
)
# Everything went great!
clean_exit = True
except EmptyPoolError:
# Didn't get a connection from the pool, no need to clean up
clean_exit = True
release_this_conn = False
raise
except (
TimeoutError,
HTTPException,
SocketError,
ProtocolError,
BaseSSLError,
SSLError,
CertificateError,
) as e:
# Discard the connection for these exceptions. It will be
# replaced during the next _get_conn() call.
clean_exit = False
def _is_ssl_error_message_from_http_proxy(ssl_error):
# We're trying to detect the message 'WRONG_VERSION_NUMBER' but
# SSLErrors are kinda all over the place when it comes to the message,
# so we try to cover our bases here!
message = " ".join(re.split("[^a-z]", str(ssl_error).lower()))
return (
"wrong version number" in message or "unknown protocol" in message
)
# Try to detect a common user error with proxies which is to
# set an HTTP proxy to be HTTPS when it should be 'http://'
# (ie {'http': 'http://proxy', 'https': 'https://proxy'})
# Instead we add a nice error message and point to a URL.
if (
isinstance(e, BaseSSLError)
and self.proxy
and _is_ssl_error_message_from_http_proxy(e)
and conn.proxy
and conn.proxy.scheme == "https"
):
e = ProxyError(
"Your proxy appears to only use HTTP and not HTTPS, "
"try changing your proxy URL to be HTTP. See: "
"https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html"
"#https-proxy-error-http-proxy",
SSLError(e),
)
elif isinstance(e, (BaseSSLError, CertificateError)):
e = SSLError(e)
elif isinstance(e, (SocketError, NewConnectionError)) and self.proxy:
e = ProxyError("Cannot connect to proxy.", e)
elif isinstance(e, (SocketError, HTTPException)):
e = ProtocolError("Connection aborted.", e)
> retries = retries.increment(
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:787:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = Retry(total=0, connect=None, read=False, redirect=None, status=None)
method = 'POST', url = '/user/credentials/cdis/access_token', response = None
error = NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8b75bd0a00>: Failed to establish a new connection: [Errno 110] Connection timed out')
_pool = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b75b450a0>
_stacktrace = <traceback object at 0x7f8b759a8c40>
def increment(
self,
method=None,
url=None,
response=None,
error=None,
_pool=None,
_stacktrace=None,
):
"""Return a new Retry object with incremented retry counters.
:param response: A response object, or None, if the server did not
return a response.
:type response: :class:`~urllib3.response.HTTPResponse`
:param Exception error: An error encountered during the request, or
None if the response was received successfully.
:return: A new ``Retry`` object.
"""
if self.total is False and error:
# Disabled, indicate to re-raise the error.
raise six.reraise(type(error), error, _stacktrace)
total = self.total
if total is not None:
total -= 1
connect = self.connect
read = self.read
redirect = self.redirect
status_count = self.status
other = self.other
cause = "unknown"
status = None
redirect_location = None
if error and self._is_connection_error(error):
# Connect retry?
if connect is False:
raise six.reraise(type(error), error, _stacktrace)
elif connect is not None:
connect -= 1
elif error and self._is_read_error(error):
# Read retry?
if read is False or not self._is_method_retryable(method):
raise six.reraise(type(error), error, _stacktrace)
elif read is not None:
read -= 1
elif error:
# Other retry?
if other is not None:
other -= 1
elif response and response.get_redirect_location():
# Redirect retry?
if redirect is not None:
redirect -= 1
cause = "too many redirects"
redirect_location = response.get_redirect_location()
status = response.status
else:
# Incrementing because of a server error like a 500 in
# status_forcelist and the given method is in the allowed_methods
cause = ResponseError.GENERIC_ERROR
if response and response.status:
if status_count is not None:
status_count -= 1
cause = ResponseError.SPECIFIC_ERROR.format(status_code=response.status)
status = response.status
history = self.history + (
RequestHistory(method, url, error, status, redirect_location),
)
new_retry = self.new(
total=total,
connect=connect,
read=read,
redirect=redirect,
status=status_count,
other=other,
history=history,
)
if new_retry.is_exhausted():
> raise MaxRetryError(_pool, url, error or ResponseError(cause))
E urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8b75bd0a00>: Failed to establish a new connection: [Errno 110] Connection timed out'))
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/retry.py:592: MaxRetryError
During handling of the above exception, another exception occurred:
client = <starlette.testclient.TestClient object at 0x7f8b75995df0>
def test_get_gen3_record(client):
UUID = "5b9ae1bd-e780-4869-a458-b3422084c480"
pass_case = {
"program": "demo1",
"project": "12L",
}
> response = client.post(f"/record/{UUID}", json=pass_case)
tests/test_api.py:103:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:635: in post
return self.request("POST", url, data=data, json=json, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:476: in request
return super().request(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:587: in request
resp = self.send(prep, **send_kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:701: in send
r = adapter.send(request, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:270: in send
raise exc
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:267: in send
portal.call(self.app, scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:283: in call
return cast(T_Retval, self.start_task_soon(func, *args).result())
/usr/lib/python3.9/concurrent/futures/_base.py:446: in result
return self.__get_result()
/usr/lib/python3.9/concurrent/futures/_base.py:391: in __get_result
raise self._exception
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:219: in _call_func
retval = await retval
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/applications.py:269: in __call__
await super().__call__(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/applications.py:124: in __call__
await self.middleware_stack(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:184: in __call__
raise exc
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:162: in __call__
await self.app(scope, receive, _send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/cors.py:84: in __call__
await self.app(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:93: in __call__
raise exc
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:82: in __call__
await self.app(scope, receive, sender)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:21: in __call__
raise e
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:18: in __call__
await self.app(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:670: in __call__
await route.handle(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:266: in handle
await self.app(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:65: in app
response = await func(request)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:227: in app
raw_response = await run_endpoint_function(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:160: in run_endpoint_function
return await dependant.call(**values)
app/main.py:218: in get_gen3_record
record = SUBMISSION.export_record(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/submission.py:348: in export_record
output = requests.get(api_url, auth=self._auth_provider).text
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:73: in get
return request("get", url, params=params, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request
return session.request(method=method, url=url, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:573: in request
prep = self.prepare_request(req)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:484: in prepare_request
p.prepare(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:372: in prepare
self.prepare_auth(auth, url)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:603: in prepare_auth
r = auth(self)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:266: in __call__
request.headers["Authorization"] = self._get_auth_value()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:378: in _get_auth_value
return "bearer " + self.get_access_token()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:367: in get_access_token
return self.refresh_access_token(endpoint)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:304: in refresh_access_token
self._access_token = get_access_token_with_key(self._refresh_token)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:75: in get_access_token_with_key
resp = requests.post(auth_url, json=api_key)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:115: in post
return request("post", url, data=data, json=json, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request
return session.request(method=method, url=url, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:587: in request
resp = self.send(prep, **send_kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:701: in send
r = adapter.send(request, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <requests.adapters.HTTPAdapter object at 0x7f8b75b45e20>
request = <PreparedRequest [POST]>, stream = False
timeout = Timeout(connect=None, read=None, total=None), verify = True
cert = None, proxies = OrderedDict()
def send(
self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None
):
"""Sends PreparedRequest object. Returns Response object.
:param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
:param stream: (optional) Whether to stream the request content.
:param timeout: (optional) How long to wait for the server to send
data before giving up, as a float, or a :ref:`(connect timeout,
read timeout) <timeouts>` tuple.
:type timeout: float or tuple or urllib3 Timeout object
:param verify: (optional) Either a boolean, in which case it controls whether
we verify the server's TLS certificate, or a string, in which case it
must be a path to a CA bundle to use
:param cert: (optional) Any user-provided SSL certificate to be trusted.
:param proxies: (optional) The proxies dictionary to apply to the request.
:rtype: requests.Response
"""
try:
conn = self.get_connection(request.url, proxies)
except LocationValueError as e:
raise InvalidURL(e, request=request)
self.cert_verify(conn, request.url, verify, cert)
url = self.request_url(request, proxies)
self.add_headers(
request,
stream=stream,
timeout=timeout,
verify=verify,
cert=cert,
proxies=proxies,
)
chunked = not (request.body is None or "Content-Length" in request.headers)
if isinstance(timeout, tuple):
try:
connect, read = timeout
timeout = TimeoutSauce(connect=connect, read=read)
except ValueError:
raise ValueError(
f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, "
f"or a single float to set both timeouts to the same value."
)
elif isinstance(timeout, TimeoutSauce):
pass
else:
timeout = TimeoutSauce(connect=timeout, read=timeout)
try:
if not chunked:
resp = conn.urlopen(
method=request.method,
url=url,
body=request.body,
headers=request.headers,
redirect=False,
assert_same_host=False,
preload_content=False,
decode_content=False,
retries=self.max_retries,
timeout=timeout,
)
# Send the request.
else:
if hasattr(conn, "proxy_pool"):
conn = conn.proxy_pool
low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)
try:
skip_host = "Host" in request.headers
low_conn.putrequest(
request.method,
url,
skip_accept_encoding=True,
skip_host=skip_host,
)
for header, value in request.headers.items():
low_conn.putheader(header, value)
low_conn.endheaders()
for i in request.body:
low_conn.send(hex(len(i))[2:].encode("utf-8"))
low_conn.send(b"\r\n")
low_conn.send(i)
low_conn.send(b"\r\n")
low_conn.send(b"0\r\n\r\n")
# Receive the response from the server
r = low_conn.getresponse()
resp = HTTPResponse.from_httplib(
r,
pool=conn,
connection=low_conn,
preload_content=False,
decode_content=False,
)
except Exception:
# If we hit any problems here, clean up the connection.
# Then, raise so that we can handle the actual exception.
low_conn.close()
raise
except (ProtocolError, OSError) as err:
raise ConnectionError(err, request=request)
except MaxRetryError as e:
if isinstance(e.reason, ConnectTimeoutError):
# TODO: Remove this in 3.0.0: see #2811
if not isinstance(e.reason, NewConnectionError):
raise ConnectTimeout(e, request=request)
if isinstance(e.reason, ResponseError):
raise RetryError(e, request=request)
if isinstance(e.reason, _ProxyError):
raise ProxyError(e, request=request)
if isinstance(e.reason, _SSLError):
# This branch is for urllib3 v1.22 and later.
raise SSLError(e, request=request)
> raise ConnectionError(e, request=request)
E requests.exceptions.ConnectionError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8b75bd0a00>: Failed to establish a new connection: [Errno 110] Connection timed out'))
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:565: ConnectionError
______________________________ test_graphql_query ______________________________
client = <starlette.testclient.TestClient object at 0x7f8b75cc2940>
def test_graphql_query(client):
DATASET_ID = "dataset-217-version-2"
pass_case = {
"node": "experiment_query",
"filter": {
"submitter_id": [DATASET_ID]
},
"search": ""
}
response = client.post("/graphql/query", json=pass_case)
result = response.json()
> assert response.status_code == 200
E assert 404 == 200
E + where 404 = <Response [404]>.status_code
tests/test_api.py:145: AssertionError
___________________________ test_graphql_pagination ____________________________
client = <starlette.testclient.TestClient object at 0x7f8b758d44c0>
def test_graphql_pagination(client):
filter_pass_case = {
"filter": {
"1": {
"node": "manifest_filter",
"filter": {
"additional_types": [
"text/vnd.abi.plot+tab-separated-values",
"text/vnd.abi.plot+csv"
]
}
},
"2": {
"node": "case_filter",
"filter": {
"species": [
"Rattus norvegicus"
]
}
},
"3": {
"node": "case_filter",
"filter": {
"sex": [
"Male"
]
}
}
},
"relation": "and"
}
response = client.post("/graphql/pagination/", json=filter_pass_case)
result = response.json()
> assert response.status_code == 200
E assert 404 == 200
E + where 404 = <Response [404]>.status_code
tests/test_api.py:188: AssertionError
_______________________ test_download_gen3_metadata_file _______________________
client = <starlette.testclient.TestClient object at 0x7f8b7619b940>
def test_download_gen3_metadata_file(client):
PROG_NAME = "demo1"
PROJ_NAME = "12L"
UUID = "22c4459b-5f4f-4e62-abd2-2aa205fe838b"
FORM = "json"
response = client.get(
f"/metadata/download/{PROG_NAME}/{PROJ_NAME}/{UUID}/{FORM}")
result = response.json()
> assert response.status_code == 200
E assert 400 == 200
E + where 400 = <Response [400]>.status_code
tests/test_api.py:285: AssertionError
_____________________________ test_experiment_node _____________________________
self = <urllib3.connection.HTTPSConnection object at 0x7f8b7611d820>
def _new_conn(self):
"""Establish a socket connection and set nodelay settings on it.
:return: New socket connection.
"""
extra_kw = {}
if self.source_address:
extra_kw["source_address"] = self.source_address
if self.socket_options:
extra_kw["socket_options"] = self.socket_options
try:
> conn = connection.create_connection(
(self._dns_host, self.port), self.timeout, **extra_kw
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:174:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
address = ('gen3.abi-ctt-ctp.cloud.edu.au', 443), timeout = None
source_address = None, socket_options = [(6, 1, 1)]
def create_connection(
address,
timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
source_address=None,
socket_options=None,
):
"""Connect to *address* and return the socket object.
Convenience function. Connect to *address* (a 2-tuple ``(host,
port)``) and return the socket object. Passing the optional
*timeout* parameter will set the timeout on the socket instance
before attempting to connect. If no *timeout* is supplied, the
global default timeout setting returned by :func:`socket.getdefaulttimeout`
is used. If *source_address* is set it must be a tuple of (host, port)
for the socket to bind as a source address before making the connection.
An host of '' or port 0 tells the OS to use the default.
"""
host, port = address
if host.startswith("["):
host = host.strip("[]")
err = None
# Using the value from allowed_gai_family() in the context of getaddrinfo lets
# us select whether to work with IPv4 DNS records, IPv6 records, or both.
# The original create_connection function always returns all records.
family = allowed_gai_family()
try:
host.encode("idna")
except UnicodeError:
return six.raise_from(
LocationParseError(u"'%s', label empty or too long" % host), None
)
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
af, socktype, proto, canonname, sa = res
sock = None
try:
sock = socket.socket(af, socktype, proto)
# If provided, set socket level options before connecting.
_set_socket_options(sock, socket_options)
if timeout is not socket._GLOBAL_DEFAULT_TIMEOUT:
sock.settimeout(timeout)
if source_address:
sock.bind(source_address)
sock.connect(sa)
return sock
except socket.error as e:
err = e
if sock is not None:
sock.close()
sock = None
if err is not None:
> raise err
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/connection.py:95:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
address = ('gen3.abi-ctt-ctp.cloud.edu.au', 443), timeout = None
source_address = None, socket_options = [(6, 1, 1)]
def create_connection(
address,
timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
source_address=None,
socket_options=None,
):
"""Connect to *address* and return the socket object.
Convenience function. Connect to *address* (a 2-tuple ``(host,
port)``) and return the socket object. Passing the optional
*timeout* parameter will set the timeout on the socket instance
before attempting to connect. If no *timeout* is supplied, the
global default timeout setting returned by :func:`socket.getdefaulttimeout`
is used. If *source_address* is set it must be a tuple of (host, port)
for the socket to bind as a source address before making the connection.
An host of '' or port 0 tells the OS to use the default.
"""
host, port = address
if host.startswith("["):
host = host.strip("[]")
err = None
# Using the value from allowed_gai_family() in the context of getaddrinfo lets
# us select whether to work with IPv4 DNS records, IPv6 records, or both.
# The original create_connection function always returns all records.
family = allowed_gai_family()
try:
host.encode("idna")
except UnicodeError:
return six.raise_from(
LocationParseError(u"'%s', label empty or too long" % host), None
)
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
af, socktype, proto, canonname, sa = res
sock = None
try:
sock = socket.socket(af, socktype, proto)
# If provided, set socket level options before connecting.
_set_socket_options(sock, socket_options)
if timeout is not socket._GLOBAL_DEFAULT_TIMEOUT:
sock.settimeout(timeout)
if source_address:
sock.bind(source_address)
> sock.connect(sa)
E TimeoutError: [Errno 110] Connection timed out
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/connection.py:85: TimeoutError
During handling of the above exception, another exception occurred:
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b7611d040>
method = 'POST', url = '/user/credentials/cdis/access_token'
body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...vPrcq8DT5FB92JLhLStE4nkB9Ysm96h77xFJ44rGq3TODjZaY6Tdj1JgHQrf8BoV2Q", "key_id": "cc7139d4-a9a2-45db-98f8-309087b6b674"}'
headers = {'User-Agent': 'python-requests/2.28.2', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}
retries = Retry(total=0, connect=None, read=False, redirect=None, status=None)
redirect = False, assert_same_host = False
timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None
release_conn = False, chunked = False, body_pos = None
response_kw = {'decode_content': False, 'preload_content': False}
parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None)
destination_scheme = None, conn = None, release_this_conn = True
http_tunnel_required = False, err = None, clean_exit = False
def urlopen(
self,
method,
url,
body=None,
headers=None,
retries=None,
redirect=True,
assert_same_host=True,
timeout=_Default,
pool_timeout=None,
release_conn=None,
chunked=False,
body_pos=None,
**response_kw
):
"""
Get a connection from the pool and perform an HTTP request. This is the
lowest level call for making a request, so you'll need to specify all
the raw details.
.. note::
More commonly, it's appropriate to use a convenience method provided
by :class:`.RequestMethods`, such as :meth:`request`.
.. note::
`release_conn` will only behave as expected if
`preload_content=False` because we want to make
`preload_content=False` the default behaviour someday soon without
breaking backwards compatibility.
:param method:
HTTP request method (such as GET, POST, PUT, etc.)
:param url:
The URL to perform the request on.
:param body:
Data to send in the request body, either :class:`str`, :class:`bytes`,
an iterable of :class:`str`/:class:`bytes`, or a file-like object.
:param headers:
Dictionary of custom headers to send, such as User-Agent,
If-None-Match, etc. If None, pool headers are used. If provided,
these headers completely replace any pool-specific headers.
:param retries:
Configure the number of retries to allow before raising a
:class:`~urllib3.exceptions.MaxRetryError` exception.
Pass ``None`` to retry until you receive a response. Pass a
:class:`~urllib3.util.retry.Retry` object for fine-grained control
over different types of retries.
Pass an integer number to retry connection errors that many times,
but no other types of errors. Pass zero to never retry.
If ``False``, then retries are disabled and any exception is raised
immediately. Also, instead of raising a MaxRetryError on redirects,
the redirect response will be returned.
:type retries: :class:`~urllib3.util.retry.Retry`, False, or an int.
:param redirect:
If True, automatically handle redirects (status codes 301, 302,
303, 307, 308). Each redirect counts as a retry. Disabling retries
will disable redirect, too.
:param assert_same_host:
If ``True``, will make sure that the host of the pool requests is
consistent else will raise HostChangedError. When ``False``, you can
use the pool on an HTTP proxy and request foreign hosts.
:param timeout:
If specified, overrides the default timeout for this one
request. It may be a float (in seconds) or an instance of
:class:`urllib3.util.Timeout`.
:param pool_timeout:
If set and the pool is set to block=True, then this method will
block for ``pool_timeout`` seconds and raise EmptyPoolError if no
connection is available within the time period.
:param release_conn:
If False, then the urlopen call will not release the connection
back into the pool once a response is received (but will release if
you read the entire contents of the response such as when
`preload_content=True`). This is useful if you're not preloading
the response's content immediately. You will need to call
``r.release_conn()`` on the response ``r`` to return the connection
back into the pool. If None, it takes the value of
``response_kw.get('preload_content', True)``.
:param chunked:
If True, urllib3 will send the body using chunked transfer
encoding. Otherwise, urllib3 will send the body using the standard
content-length form. Defaults to False.
:param int body_pos:
Position to seek to in file-like body in the event of a retry or
redirect. Typically this won't need to be set because urllib3 will
auto-populate the value when needed.
:param \\**response_kw:
Additional parameters are passed to
:meth:`urllib3.response.HTTPResponse.from_httplib`
"""
parsed_url = parse_url(url)
destination_scheme = parsed_url.scheme
if headers is None:
headers = self.headers
if not isinstance(retries, Retry):
retries = Retry.from_int(retries, redirect=redirect, default=self.retries)
if release_conn is None:
release_conn = response_kw.get("preload_content", True)
# Check host
if assert_same_host and not self.is_same_host(url):
raise HostChangedError(self, url, retries)
# Ensure that the URL we're connecting to is properly encoded
if url.startswith("/"):
url = six.ensure_str(_encode_target(url))
else:
url = six.ensure_str(parsed_url.url)
conn = None
# Track whether `conn` needs to be released before
# returning/raising/recursing. Update this variable if necessary, and
# leave `release_conn` constant throughout the function. That way, if
# the function recurses, the original value of `release_conn` will be
# passed down into the recursive call, and its value will be respected.
#
# See issue #651 [1] for details.
#
# [1] <https://github.com/urllib3/urllib3/issues/651>
release_this_conn = release_conn
http_tunnel_required = connection_requires_http_tunnel(
self.proxy, self.proxy_config, destination_scheme
)
# Merge the proxy headers. Only done when not using HTTP CONNECT. We
# have to copy the headers dict so we can safely change it without those
# changes being reflected in anyone else's copy.
if not http_tunnel_required:
headers = headers.copy()
headers.update(self.proxy_headers)
# Must keep the exception bound to a separate variable or else Python 3
# complains about UnboundLocalError.
err = None
# Keep track of whether we cleanly exited the except block. This
# ensures we do proper cleanup in finally.
clean_exit = False
# Rewind body position, if needed. Record current position
# for future rewinds in the event of a redirect/retry.
body_pos = set_file_position(body, body_pos)
try:
# Request a connection from the queue.
timeout_obj = self._get_timeout(timeout)
conn = self._get_conn(timeout=pool_timeout)
conn.timeout = timeout_obj.connect_timeout
is_new_proxy_conn = self.proxy is not None and not getattr(
conn, "sock", None
)
if is_new_proxy_conn and http_tunnel_required:
self._prepare_proxy(conn)
# Make the request on the httplib connection object.
> httplib_response = self._make_request(
conn,
method,
url,
timeout=timeout_obj,
body=body,
headers=headers,
chunked=chunked,
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:703:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b7611d040>
conn = <urllib3.connection.HTTPSConnection object at 0x7f8b7611d820>
method = 'POST', url = '/user/credentials/cdis/access_token'
timeout = Timeout(connect=None, read=None, total=None), chunked = False
httplib_request_kw = {'body': b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwd...p, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}}
timeout_obj = Timeout(connect=None, read=None, total=None)
def _make_request(
self, conn, method, url, timeout=_Default, chunked=False, **httplib_request_kw
):
"""
Perform a request on a given urllib connection object taken from our
pool.
:param conn:
a connection from one of our connection pools
:param timeout:
Socket timeout in seconds for the request. This can be a
float or integer, which will set the same timeout value for
the socket connect and the socket read, or an instance of
:class:`urllib3.util.Timeout`, which gives you more fine-grained
control over your timeouts.
"""
self.num_requests += 1
timeout_obj = self._get_timeout(timeout)
timeout_obj.start_connect()
conn.timeout = timeout_obj.connect_timeout
# Trigger any extra validation we need to do.
try:
> self._validate_conn(conn)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:386:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b7611d040>
conn = <urllib3.connection.HTTPSConnection object at 0x7f8b7611d820>
def _validate_conn(self, conn):
"""
Called right before a request is made, after the socket is created.
"""
super(HTTPSConnectionPool, self)._validate_conn(conn)
# Force connect early to allow us to validate the connection.
if not getattr(conn, "sock", None): # AppEngine might not have `.sock`
> conn.connect()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:1042:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connection.HTTPSConnection object at 0x7f8b7611d820>
def connect(self):
# Add certificate verification
> self.sock = conn = self._new_conn()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:358:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connection.HTTPSConnection object at 0x7f8b7611d820>
def _new_conn(self):
"""Establish a socket connection and set nodelay settings on it.
:return: New socket connection.
"""
extra_kw = {}
if self.source_address:
extra_kw["source_address"] = self.source_address
if self.socket_options:
extra_kw["socket_options"] = self.socket_options
try:
conn = connection.create_connection(
(self._dns_host, self.port), self.timeout, **extra_kw
)
except SocketTimeout:
raise ConnectTimeoutError(
self,
"Connection to %s timed out. (connect timeout=%s)"
% (self.host, self.timeout),
)
except SocketError as e:
> raise NewConnectionError(
self, "Failed to establish a new connection: %s" % e
)
E urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f8b7611d820>: Failed to establish a new connection: [Errno 110] Connection timed out
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:186: NewConnectionError
During handling of the above exception, another exception occurred:
self = <requests.adapters.HTTPAdapter object at 0x7f8b75946f40>
request = <PreparedRequest [POST]>, stream = False
timeout = Timeout(connect=None, read=None, total=None), verify = True
cert = None, proxies = OrderedDict()
def send(
self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None
):
"""Sends PreparedRequest object. Returns Response object.
:param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
:param stream: (optional) Whether to stream the request content.
:param timeout: (optional) How long to wait for the server to send
data before giving up, as a float, or a :ref:`(connect timeout,
read timeout) <timeouts>` tuple.
:type timeout: float or tuple or urllib3 Timeout object
:param verify: (optional) Either a boolean, in which case it controls whether
we verify the server's TLS certificate, or a string, in which case it
must be a path to a CA bundle to use
:param cert: (optional) Any user-provided SSL certificate to be trusted.
:param proxies: (optional) The proxies dictionary to apply to the request.
:rtype: requests.Response
"""
try:
conn = self.get_connection(request.url, proxies)
except LocationValueError as e:
raise InvalidURL(e, request=request)
self.cert_verify(conn, request.url, verify, cert)
url = self.request_url(request, proxies)
self.add_headers(
request,
stream=stream,
timeout=timeout,
verify=verify,
cert=cert,
proxies=proxies,
)
chunked = not (request.body is None or "Content-Length" in request.headers)
if isinstance(timeout, tuple):
try:
connect, read = timeout
timeout = TimeoutSauce(connect=connect, read=read)
except ValueError:
raise ValueError(
f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, "
f"or a single float to set both timeouts to the same value."
)
elif isinstance(timeout, TimeoutSauce):
pass
else:
timeout = TimeoutSauce(connect=timeout, read=timeout)
try:
if not chunked:
> resp = conn.urlopen(
method=request.method,
url=url,
body=request.body,
headers=request.headers,
redirect=False,
assert_same_host=False,
preload_content=False,
decode_content=False,
retries=self.max_retries,
timeout=timeout,
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:489:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b7611d040>
method = 'POST', url = '/user/credentials/cdis/access_token'
body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...vPrcq8DT5FB92JLhLStE4nkB9Ysm96h77xFJ44rGq3TODjZaY6Tdj1JgHQrf8BoV2Q", "key_id": "cc7139d4-a9a2-45db-98f8-309087b6b674"}'
headers = {'User-Agent': 'python-requests/2.28.2', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}
retries = Retry(total=0, connect=None, read=False, redirect=None, status=None)
redirect = False, assert_same_host = False
timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None
release_conn = False, chunked = False, body_pos = None
response_kw = {'decode_content': False, 'preload_content': False}
parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None)
destination_scheme = None, conn = None, release_this_conn = True
http_tunnel_required = False, err = None, clean_exit = False
def urlopen(
self,
method,
url,
body=None,
headers=None,
retries=None,
redirect=True,
assert_same_host=True,
timeout=_Default,
pool_timeout=None,
release_conn=None,
chunked=False,
body_pos=None,
**response_kw
):
"""
Get a connection from the pool and perform an HTTP request. This is the
lowest level call for making a request, so you'll need to specify all
the raw details.
.. note::
More commonly, it's appropriate to use a convenience method provided
by :class:`.RequestMethods`, such as :meth:`request`.
.. note::
`release_conn` will only behave as expected if
`preload_content=False` because we want to make
`preload_content=False` the default behaviour someday soon without
breaking backwards compatibility.
:param method:
HTTP request method (such as GET, POST, PUT, etc.)
:param url:
The URL to perform the request on.
:param body:
Data to send in the request body, either :class:`str`, :class:`bytes`,
an iterable of :class:`str`/:class:`bytes`, or a file-like object.
:param headers:
Dictionary of custom headers to send, such as User-Agent,
If-None-Match, etc. If None, pool headers are used. If provided,
these headers completely replace any pool-specific headers.
:param retries:
Configure the number of retries to allow before raising a
:class:`~urllib3.exceptions.MaxRetryError` exception.
Pass ``None`` to retry until you receive a response. Pass a
:class:`~urllib3.util.retry.Retry` object for fine-grained control
over different types of retries.
Pass an integer number to retry connection errors that many times,
but no other types of errors. Pass zero to never retry.
If ``False``, then retries are disabled and any exception is raised
immediately. Also, instead of raising a MaxRetryError on redirects,
the redirect response will be returned.
:type retries: :class:`~urllib3.util.retry.Retry`, False, or an int.
:param redirect:
If True, automatically handle redirects (status codes 301, 302,
303, 307, 308). Each redirect counts as a retry. Disabling retries
will disable redirect, too.
:param assert_same_host:
If ``True``, will make sure that the host of the pool requests is
consistent else will raise HostChangedError. When ``False``, you can
use the pool on an HTTP proxy and request foreign hosts.
:param timeout:
If specified, overrides the default timeout for this one
request. It may be a float (in seconds) or an instance of
:class:`urllib3.util.Timeout`.
:param pool_timeout:
If set and the pool is set to block=True, then this method will
block for ``pool_timeout`` seconds and raise EmptyPoolError if no
connection is available within the time period.
:param release_conn:
If False, then the urlopen call will not release the connection
back into the pool once a response is received (but will release if
you read the entire contents of the response such as when
`preload_content=True`). This is useful if you're not preloading
the response's content immediately. You will need to call
``r.release_conn()`` on the response ``r`` to return the connection
back into the pool. If None, it takes the value of
``response_kw.get('preload_content', True)``.
:param chunked:
If True, urllib3 will send the body using chunked transfer
encoding. Otherwise, urllib3 will send the body using the standard
content-length form. Defaults to False.
:param int body_pos:
Position to seek to in file-like body in the event of a retry or
redirect. Typically this won't need to be set because urllib3 will
auto-populate the value when needed.
:param \\**response_kw:
Additional parameters are passed to
:meth:`urllib3.response.HTTPResponse.from_httplib`
"""
parsed_url = parse_url(url)
destination_scheme = parsed_url.scheme
if headers is None:
headers = self.headers
if not isinstance(retries, Retry):
retries = Retry.from_int(retries, redirect=redirect, default=self.retries)
if release_conn is None:
release_conn = response_kw.get("preload_content", True)
# Check host
if assert_same_host and not self.is_same_host(url):
raise HostChangedError(self, url, retries)
# Ensure that the URL we're connecting to is properly encoded
if url.startswith("/"):
url = six.ensure_str(_encode_target(url))
else:
url = six.ensure_str(parsed_url.url)
conn = None
# Track whether `conn` needs to be released before
# returning/raising/recursing. Update this variable if necessary, and
# leave `release_conn` constant throughout the function. That way, if
# the function recurses, the original value of `release_conn` will be
# passed down into the recursive call, and its value will be respected.
#
# See issue #651 [1] for details.
#
# [1] <https://github.com/urllib3/urllib3/issues/651>
release_this_conn = release_conn
http_tunnel_required = connection_requires_http_tunnel(
self.proxy, self.proxy_config, destination_scheme
)
# Merge the proxy headers. Only done when not using HTTP CONNECT. We
# have to copy the headers dict so we can safely change it without those
# changes being reflected in anyone else's copy.
if not http_tunnel_required:
headers = headers.copy()
headers.update(self.proxy_headers)
# Must keep the exception bound to a separate variable or else Python 3
# complains about UnboundLocalError.
err = None
# Keep track of whether we cleanly exited the except block. This
# ensures we do proper cleanup in finally.
clean_exit = False
# Rewind body position, if needed. Record current position
# for future rewinds in the event of a redirect/retry.
body_pos = set_file_position(body, body_pos)
try:
# Request a connection from the queue.
timeout_obj = self._get_timeout(timeout)
conn = self._get_conn(timeout=pool_timeout)
conn.timeout = timeout_obj.connect_timeout
is_new_proxy_conn = self.proxy is not None and not getattr(
conn, "sock", None
)
if is_new_proxy_conn and http_tunnel_required:
self._prepare_proxy(conn)
# Make the request on the httplib connection object.
httplib_response = self._make_request(
conn,
method,
url,
timeout=timeout_obj,
body=body,
headers=headers,
chunked=chunked,
)
# If we're going to release the connection in ``finally:``, then
# the response doesn't need to know about the connection. Otherwise
# it will also try to release it and we'll have a double-release
# mess.
response_conn = conn if not release_conn else None
# Pass method to Response for length checking
response_kw["request_method"] = method
# Import httplib's response into our own wrapper object
response = self.ResponseCls.from_httplib(
httplib_response,
pool=self,
connection=response_conn,
retries=retries,
**response_kw
)
# Everything went great!
clean_exit = True
except EmptyPoolError:
# Didn't get a connection from the pool, no need to clean up
clean_exit = True
release_this_conn = False
raise
except (
TimeoutError,
HTTPException,
SocketError,
ProtocolError,
BaseSSLError,
SSLError,
CertificateError,
) as e:
# Discard the connection for these exceptions. It will be
# replaced during the next _get_conn() call.
clean_exit = False
def _is_ssl_error_message_from_http_proxy(ssl_error):
# We're trying to detect the message 'WRONG_VERSION_NUMBER' but
# SSLErrors are kinda all over the place when it comes to the message,
# so we try to cover our bases here!
message = " ".join(re.split("[^a-z]", str(ssl_error).lower()))
return (
"wrong version number" in message or "unknown protocol" in message
)
# Try to detect a common user error with proxies which is to
# set an HTTP proxy to be HTTPS when it should be 'http://'
# (ie {'http': 'http://proxy', 'https': 'https://proxy'})
# Instead we add a nice error message and point to a URL.
if (
isinstance(e, BaseSSLError)
and self.proxy
and _is_ssl_error_message_from_http_proxy(e)
and conn.proxy
and conn.proxy.scheme == "https"
):
e = ProxyError(
"Your proxy appears to only use HTTP and not HTTPS, "
"try changing your proxy URL to be HTTP. See: "
"https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html"
"#https-proxy-error-http-proxy",
SSLError(e),
)
elif isinstance(e, (BaseSSLError, CertificateError)):
e = SSLError(e)
elif isinstance(e, (SocketError, NewConnectionError)) and self.proxy:
e = ProxyError("Cannot connect to proxy.", e)
elif isinstance(e, (SocketError, HTTPException)):
e = ProtocolError("Connection aborted.", e)
> retries = retries.increment(
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:787:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = Retry(total=0, connect=None, read=False, redirect=None, status=None)
method = 'POST', url = '/user/credentials/cdis/access_token', response = None
error = NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8b7611d820>: Failed to establish a new connection: [Errno 110] Connection timed out')
_pool = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b7611d040>
_stacktrace = <traceback object at 0x7f8b76ba6680>
def increment(
self,
method=None,
url=None,
response=None,
error=None,
_pool=None,
_stacktrace=None,
):
"""Return a new Retry object with incremented retry counters.
:param response: A response object, or None, if the server did not
return a response.
:type response: :class:`~urllib3.response.HTTPResponse`
:param Exception error: An error encountered during the request, or
None if the response was received successfully.
:return: A new ``Retry`` object.
"""
if self.total is False and error:
# Disabled, indicate to re-raise the error.
raise six.reraise(type(error), error, _stacktrace)
total = self.total
if total is not None:
total -= 1
connect = self.connect
read = self.read
redirect = self.redirect
status_count = self.status
other = self.other
cause = "unknown"
status = None
redirect_location = None
if error and self._is_connection_error(error):
# Connect retry?
if connect is False:
raise six.reraise(type(error), error, _stacktrace)
elif connect is not None:
connect -= 1
elif error and self._is_read_error(error):
# Read retry?
if read is False or not self._is_method_retryable(method):
raise six.reraise(type(error), error, _stacktrace)
elif read is not None:
read -= 1
elif error:
# Other retry?
if other is not None:
other -= 1
elif response and response.get_redirect_location():
# Redirect retry?
if redirect is not None:
redirect -= 1
cause = "too many redirects"
redirect_location = response.get_redirect_location()
status = response.status
else:
# Incrementing because of a server error like a 500 in
# status_forcelist and the given method is in the allowed_methods
cause = ResponseError.GENERIC_ERROR
if response and response.status:
if status_count is not None:
status_count -= 1
cause = ResponseError.SPECIFIC_ERROR.format(status_code=response.status)
status = response.status
history = self.history + (
RequestHistory(method, url, error, status, redirect_location),
)
new_retry = self.new(
total=total,
connect=connect,
read=read,
redirect=redirect,
status=status_count,
other=other,
history=history,
)
if new_retry.is_exhausted():
> raise MaxRetryError(_pool, url, error or ResponseError(cause))
E urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8b7611d820>: Failed to establish a new connection: [Errno 110] Connection timed out'))
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/retry.py:592: MaxRetryError
During handling of the above exception, another exception occurred:
client = <starlette.testclient.TestClient object at 0x7f8b75da56d0>
def test_experiment_node(client):
UUID = "22c4459b-5f4f-4e62-abd2-2aa205fe838b"
payload = {
"program": "demo1",
"project": "12L",
}
> response = client.post(
f"/record/{UUID}", json=payload)
tests/test_data.py:22:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:635: in post
return self.request("POST", url, data=data, json=json, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:476: in request
return super().request(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:587: in request
resp = self.send(prep, **send_kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:701: in send
r = adapter.send(request, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:270: in send
raise exc
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:267: in send
portal.call(self.app, scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:283: in call
return cast(T_Retval, self.start_task_soon(func, *args).result())
/usr/lib/python3.9/concurrent/futures/_base.py:446: in result
return self.__get_result()
/usr/lib/python3.9/concurrent/futures/_base.py:391: in __get_result
raise self._exception
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:219: in _call_func
retval = await retval
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/applications.py:269: in __call__
await super().__call__(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/applications.py:124: in __call__
await self.middleware_stack(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:184: in __call__
raise exc
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:162: in __call__
await self.app(scope, receive, _send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/cors.py:84: in __call__
await self.app(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:93: in __call__
raise exc
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:82: in __call__
await self.app(scope, receive, sender)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:21: in __call__
raise e
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:18: in __call__
await self.app(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:670: in __call__
await route.handle(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:266: in handle
await self.app(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:65: in app
response = await func(request)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:227: in app
raw_response = await run_endpoint_function(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:160: in run_endpoint_function
return await dependant.call(**values)
app/main.py:218: in get_gen3_record
record = SUBMISSION.export_record(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/submission.py:348: in export_record
output = requests.get(api_url, auth=self._auth_provider).text
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:73: in get
return request("get", url, params=params, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request
return session.request(method=method, url=url, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:573: in request
prep = self.prepare_request(req)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:484: in prepare_request
p.prepare(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:372: in prepare
self.prepare_auth(auth, url)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:603: in prepare_auth
r = auth(self)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:266: in __call__
request.headers["Authorization"] = self._get_auth_value()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:378: in _get_auth_value
return "bearer " + self.get_access_token()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:367: in get_access_token
return self.refresh_access_token(endpoint)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:304: in refresh_access_token
self._access_token = get_access_token_with_key(self._refresh_token)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:75: in get_access_token_with_key
resp = requests.post(auth_url, json=api_key)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:115: in post
return request("post", url, data=data, json=json, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request
return session.request(method=method, url=url, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:587: in request
resp = self.send(prep, **send_kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:701: in send
r = adapter.send(request, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <requests.adapters.HTTPAdapter object at 0x7f8b75946f40>
request = <PreparedRequest [POST]>, stream = False
timeout = Timeout(connect=None, read=None, total=None), verify = True
cert = None, proxies = OrderedDict()
def send(
self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None
):
"""Sends PreparedRequest object. Returns Response object.
:param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
:param stream: (optional) Whether to stream the request content.
:param timeout: (optional) How long to wait for the server to send
data before giving up, as a float, or a :ref:`(connect timeout,
read timeout) <timeouts>` tuple.
:type timeout: float or tuple or urllib3 Timeout object
:param verify: (optional) Either a boolean, in which case it controls whether
we verify the server's TLS certificate, or a string, in which case it
must be a path to a CA bundle to use
:param cert: (optional) Any user-provided SSL certificate to be trusted.
:param proxies: (optional) The proxies dictionary to apply to the request.
:rtype: requests.Response
"""
try:
conn = self.get_connection(request.url, proxies)
except LocationValueError as e:
raise InvalidURL(e, request=request)
self.cert_verify(conn, request.url, verify, cert)
url = self.request_url(request, proxies)
self.add_headers(
request,
stream=stream,
timeout=timeout,
verify=verify,
cert=cert,
proxies=proxies,
)
chunked = not (request.body is None or "Content-Length" in request.headers)
if isinstance(timeout, tuple):
try:
connect, read = timeout
timeout = TimeoutSauce(connect=connect, read=read)
except ValueError:
raise ValueError(
f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, "
f"or a single float to set both timeouts to the same value."
)
elif isinstance(timeout, TimeoutSauce):
pass
else:
timeout = TimeoutSauce(connect=timeout, read=timeout)
try:
if not chunked:
resp = conn.urlopen(
method=request.method,
url=url,
body=request.body,
headers=request.headers,
redirect=False,
assert_same_host=False,
preload_content=False,
decode_content=False,
retries=self.max_retries,
timeout=timeout,
)
# Send the request.
else:
if hasattr(conn, "proxy_pool"):
conn = conn.proxy_pool
low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)
try:
skip_host = "Host" in request.headers
low_conn.putrequest(
request.method,
url,
skip_accept_encoding=True,
skip_host=skip_host,
)
for header, value in request.headers.items():
low_conn.putheader(header, value)
low_conn.endheaders()
for i in request.body:
low_conn.send(hex(len(i))[2:].encode("utf-8"))
low_conn.send(b"\r\n")
low_conn.send(i)
low_conn.send(b"\r\n")
low_conn.send(b"0\r\n\r\n")
# Receive the response from the server
r = low_conn.getresponse()
resp = HTTPResponse.from_httplib(
r,
pool=conn,
connection=low_conn,
preload_content=False,
decode_content=False,
)
except Exception:
# If we hit any problems here, clean up the connection.
# Then, raise so that we can handle the actual exception.
low_conn.close()
raise
except (ProtocolError, OSError) as err:
raise ConnectionError(err, request=request)
except MaxRetryError as e:
if isinstance(e.reason, ConnectTimeoutError):
# TODO: Remove this in 3.0.0: see #2811
if not isinstance(e.reason, NewConnectionError):
raise ConnectTimeout(e, request=request)
if isinstance(e.reason, ResponseError):
raise RetryError(e, request=request)
if isinstance(e.reason, _ProxyError):
raise ProxyError(e, request=request)
if isinstance(e.reason, _SSLError):
# This branch is for urllib3 v1.22 and later.
raise SSLError(e, request=request)
> raise ConnectionError(e, request=request)
E requests.exceptions.ConnectionError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8b7611d820>: Failed to establish a new connection: [Errno 110] Connection timed out'))
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:565: ConnectionError
________________________ test_dataset_description_node _________________________
self = <urllib3.connection.HTTPSConnection object at 0x7f8b75a6cac0>
def _new_conn(self):
"""Establish a socket connection and set nodelay settings on it.
:return: New socket connection.
"""
extra_kw = {}
if self.source_address:
extra_kw["source_address"] = self.source_address
if self.socket_options:
extra_kw["socket_options"] = self.socket_options
try:
> conn = connection.create_connection(
(self._dns_host, self.port), self.timeout, **extra_kw
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:174:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
address = ('gen3.abi-ctt-ctp.cloud.edu.au', 443), timeout = None
source_address = None, socket_options = [(6, 1, 1)]
def create_connection(
address,
timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
source_address=None,
socket_options=None,
):
"""Connect to *address* and return the socket object.
Convenience function. Connect to *address* (a 2-tuple ``(host,
port)``) and return the socket object. Passing the optional
*timeout* parameter will set the timeout on the socket instance
before attempting to connect. If no *timeout* is supplied, the
global default timeout setting returned by :func:`socket.getdefaulttimeout`
is used. If *source_address* is set it must be a tuple of (host, port)
for the socket to bind as a source address before making the connection.
An host of '' or port 0 tells the OS to use the default.
"""
host, port = address
if host.startswith("["):
host = host.strip("[]")
err = None
# Using the value from allowed_gai_family() in the context of getaddrinfo lets
# us select whether to work with IPv4 DNS records, IPv6 records, or both.
# The original create_connection function always returns all records.
family = allowed_gai_family()
try:
host.encode("idna")
except UnicodeError:
return six.raise_from(
LocationParseError(u"'%s', label empty or too long" % host), None
)
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
af, socktype, proto, canonname, sa = res
sock = None
try:
sock = socket.socket(af, socktype, proto)
# If provided, set socket level options before connecting.
_set_socket_options(sock, socket_options)
if timeout is not socket._GLOBAL_DEFAULT_TIMEOUT:
sock.settimeout(timeout)
if source_address:
sock.bind(source_address)
sock.connect(sa)
return sock
except socket.error as e:
err = e
if sock is not None:
sock.close()
sock = None
if err is not None:
> raise err
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/connection.py:95:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
address = ('gen3.abi-ctt-ctp.cloud.edu.au', 443), timeout = None
source_address = None, socket_options = [(6, 1, 1)]
def create_connection(
address,
timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
source_address=None,
socket_options=None,
):
"""Connect to *address* and return the socket object.
Convenience function. Connect to *address* (a 2-tuple ``(host,
port)``) and return the socket object. Passing the optional
*timeout* parameter will set the timeout on the socket instance
before attempting to connect. If no *timeout* is supplied, the
global default timeout setting returned by :func:`socket.getdefaulttimeout`
is used. If *source_address* is set it must be a tuple of (host, port)
for the socket to bind as a source address before making the connection.
An host of '' or port 0 tells the OS to use the default.
"""
host, port = address
if host.startswith("["):
host = host.strip("[]")
err = None
# Using the value from allowed_gai_family() in the context of getaddrinfo lets
# us select whether to work with IPv4 DNS records, IPv6 records, or both.
# The original create_connection function always returns all records.
family = allowed_gai_family()
try:
host.encode("idna")
except UnicodeError:
return six.raise_from(
LocationParseError(u"'%s', label empty or too long" % host), None
)
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
af, socktype, proto, canonname, sa = res
sock = None
try:
sock = socket.socket(af, socktype, proto)
# If provided, set socket level options before connecting.
_set_socket_options(sock, socket_options)
if timeout is not socket._GLOBAL_DEFAULT_TIMEOUT:
sock.settimeout(timeout)
if source_address:
sock.bind(source_address)
> sock.connect(sa)
E TimeoutError: [Errno 110] Connection timed out
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/connection.py:85: TimeoutError
During handling of the above exception, another exception occurred:
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b75a6ce20>
method = 'POST', url = '/user/credentials/cdis/access_token'
body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...vPrcq8DT5FB92JLhLStE4nkB9Ysm96h77xFJ44rGq3TODjZaY6Tdj1JgHQrf8BoV2Q", "key_id": "cc7139d4-a9a2-45db-98f8-309087b6b674"}'
headers = {'User-Agent': 'python-requests/2.28.2', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}
retries = Retry(total=0, connect=None, read=False, redirect=None, status=None)
redirect = False, assert_same_host = False
timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None
release_conn = False, chunked = False, body_pos = None
response_kw = {'decode_content': False, 'preload_content': False}
parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None)
destination_scheme = None, conn = None, release_this_conn = True
http_tunnel_required = False, err = None, clean_exit = False
def urlopen(
self,
method,
url,
body=None,
headers=None,
retries=None,
redirect=True,
assert_same_host=True,
timeout=_Default,
pool_timeout=None,
release_conn=None,
chunked=False,
body_pos=None,
**response_kw
):
"""
Get a connection from the pool and perform an HTTP request. This is the
lowest level call for making a request, so you'll need to specify all
the raw details.
.. note::
More commonly, it's appropriate to use a convenience method provided
by :class:`.RequestMethods`, such as :meth:`request`.
.. note::
`release_conn` will only behave as expected if
`preload_content=False` because we want to make
`preload_content=False` the default behaviour someday soon without
breaking backwards compatibility.
:param method:
HTTP request method (such as GET, POST, PUT, etc.)
:param url:
The URL to perform the request on.
:param body:
Data to send in the request body, either :class:`str`, :class:`bytes`,
an iterable of :class:`str`/:class:`bytes`, or a file-like object.
:param headers:
Dictionary of custom headers to send, such as User-Agent,
If-None-Match, etc. If None, pool headers are used. If provided,
these headers completely replace any pool-specific headers.
:param retries:
Configure the number of retries to allow before raising a
:class:`~urllib3.exceptions.MaxRetryError` exception.
Pass ``None`` to retry until you receive a response. Pass a
:class:`~urllib3.util.retry.Retry` object for fine-grained control
over different types of retries.
Pass an integer number to retry connection errors that many times,
but no other types of errors. Pass zero to never retry.
If ``False``, then retries are disabled and any exception is raised
immediately. Also, instead of raising a MaxRetryError on redirects,
the redirect response will be returned.
:type retries: :class:`~urllib3.util.retry.Retry`, False, or an int.
:param redirect:
If True, automatically handle redirects (status codes 301, 302,
303, 307, 308). Each redirect counts as a retry. Disabling retries
will disable redirect, too.
:param assert_same_host:
If ``True``, will make sure that the host of the pool requests is
consistent else will raise HostChangedError. When ``False``, you can
use the pool on an HTTP proxy and request foreign hosts.
:param timeout:
If specified, overrides the default timeout for this one
request. It may be a float (in seconds) or an instance of
:class:`urllib3.util.Timeout`.
:param pool_timeout:
If set and the pool is set to block=True, then this method will
block for ``pool_timeout`` seconds and raise EmptyPoolError if no
connection is available within the time period.
:param release_conn:
If False, then the urlopen call will not release the connection
back into the pool once a response is received (but will release if
you read the entire contents of the response such as when
`preload_content=True`). This is useful if you're not preloading
the response's content immediately. You will need to call
``r.release_conn()`` on the response ``r`` to return the connection
back into the pool. If None, it takes the value of
``response_kw.get('preload_content', True)``.
:param chunked:
If True, urllib3 will send the body using chunked transfer
encoding. Otherwise, urllib3 will send the body using the standard
content-length form. Defaults to False.
:param int body_pos:
Position to seek to in file-like body in the event of a retry or
redirect. Typically this won't need to be set because urllib3 will
auto-populate the value when needed.
:param \\**response_kw:
Additional parameters are passed to
:meth:`urllib3.response.HTTPResponse.from_httplib`
"""
parsed_url = parse_url(url)
destination_scheme = parsed_url.scheme
if headers is None:
headers = self.headers
if not isinstance(retries, Retry):
retries = Retry.from_int(retries, redirect=redirect, default=self.retries)
if release_conn is None:
release_conn = response_kw.get("preload_content", True)
# Check host
if assert_same_host and not self.is_same_host(url):
raise HostChangedError(self, url, retries)
# Ensure that the URL we're connecting to is properly encoded
if url.startswith("/"):
url = six.ensure_str(_encode_target(url))
else:
url = six.ensure_str(parsed_url.url)
conn = None
# Track whether `conn` needs to be released before
# returning/raising/recursing. Update this variable if necessary, and
# leave `release_conn` constant throughout the function. That way, if
# the function recurses, the original value of `release_conn` will be
# passed down into the recursive call, and its value will be respected.
#
# See issue #651 [1] for details.
#
# [1] <https://github.com/urllib3/urllib3/issues/651>
release_this_conn = release_conn
http_tunnel_required = connection_requires_http_tunnel(
self.proxy, self.proxy_config, destination_scheme
)
# Merge the proxy headers. Only done when not using HTTP CONNECT. We
# have to copy the headers dict so we can safely change it without those
# changes being reflected in anyone else's copy.
if not http_tunnel_required:
headers = headers.copy()
headers.update(self.proxy_headers)
# Must keep the exception bound to a separate variable or else Python 3
# complains about UnboundLocalError.
err = None
# Keep track of whether we cleanly exited the except block. This
# ensures we do proper cleanup in finally.
clean_exit = False
# Rewind body position, if needed. Record current position
# for future rewinds in the event of a redirect/retry.
body_pos = set_file_position(body, body_pos)
try:
# Request a connection from the queue.
timeout_obj = self._get_timeout(timeout)
conn = self._get_conn(timeout=pool_timeout)
conn.timeout = timeout_obj.connect_timeout
is_new_proxy_conn = self.proxy is not None and not getattr(
conn, "sock", None
)
if is_new_proxy_conn and http_tunnel_required:
self._prepare_proxy(conn)
# Make the request on the httplib connection object.
> httplib_response = self._make_request(
conn,
method,
url,
timeout=timeout_obj,
body=body,
headers=headers,
chunked=chunked,
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:703:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b75a6ce20>
conn = <urllib3.connection.HTTPSConnection object at 0x7f8b75a6cac0>
method = 'POST', url = '/user/credentials/cdis/access_token'
timeout = Timeout(connect=None, read=None, total=None), chunked = False
httplib_request_kw = {'body': b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwd...p, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}}
timeout_obj = Timeout(connect=None, read=None, total=None)
def _make_request(
self, conn, method, url, timeout=_Default, chunked=False, **httplib_request_kw
):
"""
Perform a request on a given urllib connection object taken from our
pool.
:param conn:
a connection from one of our connection pools
:param timeout:
Socket timeout in seconds for the request. This can be a
float or integer, which will set the same timeout value for
the socket connect and the socket read, or an instance of
:class:`urllib3.util.Timeout`, which gives you more fine-grained
control over your timeouts.
"""
self.num_requests += 1
timeout_obj = self._get_timeout(timeout)
timeout_obj.start_connect()
conn.timeout = timeout_obj.connect_timeout
# Trigger any extra validation we need to do.
try:
> self._validate_conn(conn)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:386:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b75a6ce20>
conn = <urllib3.connection.HTTPSConnection object at 0x7f8b75a6cac0>
def _validate_conn(self, conn):
"""
Called right before a request is made, after the socket is created.
"""
super(HTTPSConnectionPool, self)._validate_conn(conn)
# Force connect early to allow us to validate the connection.
if not getattr(conn, "sock", None): # AppEngine might not have `.sock`
> conn.connect()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:1042:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connection.HTTPSConnection object at 0x7f8b75a6cac0>
def connect(self):
# Add certificate verification
> self.sock = conn = self._new_conn()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:358:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connection.HTTPSConnection object at 0x7f8b75a6cac0>
def _new_conn(self):
"""Establish a socket connection and set nodelay settings on it.
:return: New socket connection.
"""
extra_kw = {}
if self.source_address:
extra_kw["source_address"] = self.source_address
if self.socket_options:
extra_kw["socket_options"] = self.socket_options
try:
conn = connection.create_connection(
(self._dns_host, self.port), self.timeout, **extra_kw
)
except SocketTimeout:
raise ConnectTimeoutError(
self,
"Connection to %s timed out. (connect timeout=%s)"
% (self.host, self.timeout),
)
except SocketError as e:
> raise NewConnectionError(
self, "Failed to establish a new connection: %s" % e
)
E urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f8b75a6cac0>: Failed to establish a new connection: [Errno 110] Connection timed out
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:186: NewConnectionError
During handling of the above exception, another exception occurred:
self = <requests.adapters.HTTPAdapter object at 0x7f8b75a6c490>
request = <PreparedRequest [POST]>, stream = False
timeout = Timeout(connect=None, read=None, total=None), verify = True
cert = None, proxies = OrderedDict()
def send(
self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None
):
"""Sends PreparedRequest object. Returns Response object.
:param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
:param stream: (optional) Whether to stream the request content.
:param timeout: (optional) How long to wait for the server to send
data before giving up, as a float, or a :ref:`(connect timeout,
read timeout) <timeouts>` tuple.
:type timeout: float or tuple or urllib3 Timeout object
:param verify: (optional) Either a boolean, in which case it controls whether
we verify the server's TLS certificate, or a string, in which case it
must be a path to a CA bundle to use
:param cert: (optional) Any user-provided SSL certificate to be trusted.
:param proxies: (optional) The proxies dictionary to apply to the request.
:rtype: requests.Response
"""
try:
conn = self.get_connection(request.url, proxies)
except LocationValueError as e:
raise InvalidURL(e, request=request)
self.cert_verify(conn, request.url, verify, cert)
url = self.request_url(request, proxies)
self.add_headers(
request,
stream=stream,
timeout=timeout,
verify=verify,
cert=cert,
proxies=proxies,
)
chunked = not (request.body is None or "Content-Length" in request.headers)
if isinstance(timeout, tuple):
try:
connect, read = timeout
timeout = TimeoutSauce(connect=connect, read=read)
except ValueError:
raise ValueError(
f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, "
f"or a single float to set both timeouts to the same value."
)
elif isinstance(timeout, TimeoutSauce):
pass
else:
timeout = TimeoutSauce(connect=timeout, read=timeout)
try:
if not chunked:
> resp = conn.urlopen(
method=request.method,
url=url,
body=request.body,
headers=request.headers,
redirect=False,
assert_same_host=False,
preload_content=False,
decode_content=False,
retries=self.max_retries,
timeout=timeout,
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:489:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b75a6ce20>
method = 'POST', url = '/user/credentials/cdis/access_token'
body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...vPrcq8DT5FB92JLhLStE4nkB9Ysm96h77xFJ44rGq3TODjZaY6Tdj1JgHQrf8BoV2Q", "key_id": "cc7139d4-a9a2-45db-98f8-309087b6b674"}'
headers = {'User-Agent': 'python-requests/2.28.2', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}
retries = Retry(total=0, connect=None, read=False, redirect=None, status=None)
redirect = False, assert_same_host = False
timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None
release_conn = False, chunked = False, body_pos = None
response_kw = {'decode_content': False, 'preload_content': False}
parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None)
destination_scheme = None, conn = None, release_this_conn = True
http_tunnel_required = False, err = None, clean_exit = False
def urlopen(
self,
method,
url,
body=None,
headers=None,
retries=None,
redirect=True,
assert_same_host=True,
timeout=_Default,
pool_timeout=None,
release_conn=None,
chunked=False,
body_pos=None,
**response_kw
):
"""
Get a connection from the pool and perform an HTTP request. This is the
lowest level call for making a request, so you'll need to specify all
the raw details.
.. note::
More commonly, it's appropriate to use a convenience method provided
by :class:`.RequestMethods`, such as :meth:`request`.
.. note::
`release_conn` will only behave as expected if
`preload_content=False` because we want to make
`preload_content=False` the default behaviour someday soon without
breaking backwards compatibility.
:param method:
HTTP request method (such as GET, POST, PUT, etc.)
:param url:
The URL to perform the request on.
:param body:
Data to send in the request body, either :class:`str`, :class:`bytes`,
an iterable of :class:`str`/:class:`bytes`, or a file-like object.
:param headers:
Dictionary of custom headers to send, such as User-Agent,
If-None-Match, etc. If None, pool headers are used. If provided,
these headers completely replace any pool-specific headers.
:param retries:
Configure the number of retries to allow before raising a
:class:`~urllib3.exceptions.MaxRetryError` exception.
Pass ``None`` to retry until you receive a response. Pass a
:class:`~urllib3.util.retry.Retry` object for fine-grained control
over different types of retries.
Pass an integer number to retry connection errors that many times,
but no other types of errors. Pass zero to never retry.
If ``False``, then retries are disabled and any exception is raised
immediately. Also, instead of raising a MaxRetryError on redirects,
the redirect response will be returned.
:type retries: :class:`~urllib3.util.retry.Retry`, False, or an int.
:param redirect:
If True, automatically handle redirects (status codes 301, 302,
303, 307, 308). Each redirect counts as a retry. Disabling retries
will disable redirect, too.
:param assert_same_host:
If ``True``, will make sure that the host of the pool requests is
consistent else will raise HostChangedError. When ``False``, you can
use the pool on an HTTP proxy and request foreign hosts.
:param timeout:
If specified, overrides the default timeout for this one
request. It may be a float (in seconds) or an instance of
:class:`urllib3.util.Timeout`.
:param pool_timeout:
If set and the pool is set to block=True, then this method will
block for ``pool_timeout`` seconds and raise EmptyPoolError if no
connection is available within the time period.
:param release_conn:
If False, then the urlopen call will not release the connection
back into the pool once a response is received (but will release if
you read the entire contents of the response such as when
`preload_content=True`). This is useful if you're not preloading
the response's content immediately. You will need to call
``r.release_conn()`` on the response ``r`` to return the connection
back into the pool. If None, it takes the value of
``response_kw.get('preload_content', True)``.
:param chunked:
If True, urllib3 will send the body using chunked transfer
encoding. Otherwise, urllib3 will send the body using the standard
content-length form. Defaults to False.
:param int body_pos:
Position to seek to in file-like body in the event of a retry or
redirect. Typically this won't need to be set because urllib3 will
auto-populate the value when needed.
:param \\**response_kw:
Additional parameters are passed to
:meth:`urllib3.response.HTTPResponse.from_httplib`
"""
parsed_url = parse_url(url)
destination_scheme = parsed_url.scheme
if headers is None:
headers = self.headers
if not isinstance(retries, Retry):
retries = Retry.from_int(retries, redirect=redirect, default=self.retries)
if release_conn is None:
release_conn = response_kw.get("preload_content", True)
# Check host
if assert_same_host and not self.is_same_host(url):
raise HostChangedError(self, url, retries)
# Ensure that the URL we're connecting to is properly encoded
if url.startswith("/"):
url = six.ensure_str(_encode_target(url))
else:
url = six.ensure_str(parsed_url.url)
conn = None
# Track whether `conn` needs to be released before
# returning/raising/recursing. Update this variable if necessary, and
# leave `release_conn` constant throughout the function. That way, if
# the function recurses, the original value of `release_conn` will be
# passed down into the recursive call, and its value will be respected.
#
# See issue #651 [1] for details.
#
# [1] <https://github.com/urllib3/urllib3/issues/651>
release_this_conn = release_conn
http_tunnel_required = connection_requires_http_tunnel(
self.proxy, self.proxy_config, destination_scheme
)
# Merge the proxy headers. Only done when not using HTTP CONNECT. We
# have to copy the headers dict so we can safely change it without those
# changes being reflected in anyone else's copy.
if not http_tunnel_required:
headers = headers.copy()
headers.update(self.proxy_headers)
# Must keep the exception bound to a separate variable or else Python 3
# complains about UnboundLocalError.
err = None
# Keep track of whether we cleanly exited the except block. This
# ensures we do proper cleanup in finally.
clean_exit = False
# Rewind body position, if needed. Record current position
# for future rewinds in the event of a redirect/retry.
body_pos = set_file_position(body, body_pos)
try:
# Request a connection from the queue.
timeout_obj = self._get_timeout(timeout)
conn = self._get_conn(timeout=pool_timeout)
conn.timeout = timeout_obj.connect_timeout
is_new_proxy_conn = self.proxy is not None and not getattr(
conn, "sock", None
)
if is_new_proxy_conn and http_tunnel_required:
self._prepare_proxy(conn)
# Make the request on the httplib connection object.
httplib_response = self._make_request(
conn,
method,
url,
timeout=timeout_obj,
body=body,
headers=headers,
chunked=chunked,
)
# If we're going to release the connection in ``finally:``, then
# the response doesn't need to know about the connection. Otherwise
# it will also try to release it and we'll have a double-release
# mess.
response_conn = conn if not release_conn else None
# Pass method to Response for length checking
response_kw["request_method"] = method
# Import httplib's response into our own wrapper object
response = self.ResponseCls.from_httplib(
httplib_response,
pool=self,
connection=response_conn,
retries=retries,
**response_kw
)
# Everything went great!
clean_exit = True
except EmptyPoolError:
# Didn't get a connection from the pool, no need to clean up
clean_exit = True
release_this_conn = False
raise
except (
TimeoutError,
HTTPException,
SocketError,
ProtocolError,
BaseSSLError,
SSLError,
CertificateError,
) as e:
# Discard the connection for these exceptions. It will be
# replaced during the next _get_conn() call.
clean_exit = False
def _is_ssl_error_message_from_http_proxy(ssl_error):
# We're trying to detect the message 'WRONG_VERSION_NUMBER' but
# SSLErrors are kinda all over the place when it comes to the message,
# so we try to cover our bases here!
message = " ".join(re.split("[^a-z]", str(ssl_error).lower()))
return (
"wrong version number" in message or "unknown protocol" in message
)
# Try to detect a common user error with proxies which is to
# set an HTTP proxy to be HTTPS when it should be 'http://'
# (ie {'http': 'http://proxy', 'https': 'https://proxy'})
# Instead we add a nice error message and point to a URL.
if (
isinstance(e, BaseSSLError)
and self.proxy
and _is_ssl_error_message_from_http_proxy(e)
and conn.proxy
and conn.proxy.scheme == "https"
):
e = ProxyError(
"Your proxy appears to only use HTTP and not HTTPS, "
"try changing your proxy URL to be HTTP. See: "
"https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html"
"#https-proxy-error-http-proxy",
SSLError(e),
)
elif isinstance(e, (BaseSSLError, CertificateError)):
e = SSLError(e)
elif isinstance(e, (SocketError, NewConnectionError)) and self.proxy:
e = ProxyError("Cannot connect to proxy.", e)
elif isinstance(e, (SocketError, HTTPException)):
e = ProtocolError("Connection aborted.", e)
> retries = retries.increment(
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:787:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = Retry(total=0, connect=None, read=False, redirect=None, status=None)
method = 'POST', url = '/user/credentials/cdis/access_token', response = None
error = NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8b75a6cac0>: Failed to establish a new connection: [Errno 110] Connection timed out')
_pool = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b75a6ce20>
_stacktrace = <traceback object at 0x7f8b75927680>
def increment(
self,
method=None,
url=None,
response=None,
error=None,
_pool=None,
_stacktrace=None,
):
"""Return a new Retry object with incremented retry counters.
:param response: A response object, or None, if the server did not
return a response.
:type response: :class:`~urllib3.response.HTTPResponse`
:param Exception error: An error encountered during the request, or
None if the response was received successfully.
:return: A new ``Retry`` object.
"""
if self.total is False and error:
# Disabled, indicate to re-raise the error.
raise six.reraise(type(error), error, _stacktrace)
total = self.total
if total is not None:
total -= 1
connect = self.connect
read = self.read
redirect = self.redirect
status_count = self.status
other = self.other
cause = "unknown"
status = None
redirect_location = None
if error and self._is_connection_error(error):
# Connect retry?
if connect is False:
raise six.reraise(type(error), error, _stacktrace)
elif connect is not None:
connect -= 1
elif error and self._is_read_error(error):
# Read retry?
if read is False or not self._is_method_retryable(method):
raise six.reraise(type(error), error, _stacktrace)
elif read is not None:
read -= 1
elif error:
# Other retry?
if other is not None:
other -= 1
elif response and response.get_redirect_location():
# Redirect retry?
if redirect is not None:
redirect -= 1
cause = "too many redirects"
redirect_location = response.get_redirect_location()
status = response.status
else:
# Incrementing because of a server error like a 500 in
# status_forcelist and the given method is in the allowed_methods
cause = ResponseError.GENERIC_ERROR
if response and response.status:
if status_count is not None:
status_count -= 1
cause = ResponseError.SPECIFIC_ERROR.format(status_code=response.status)
status = response.status
history = self.history + (
RequestHistory(method, url, error, status, redirect_location),
)
new_retry = self.new(
total=total,
connect=connect,
read=read,
redirect=redirect,
status=status_count,
other=other,
history=history,
)
if new_retry.is_exhausted():
> raise MaxRetryError(_pool, url, error or ResponseError(cause))
E urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8b75a6cac0>: Failed to establish a new connection: [Errno 110] Connection timed out'))
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/retry.py:592: MaxRetryError
During handling of the above exception, another exception occurred:
client = <starlette.testclient.TestClient object at 0x7f8b7ee4eb50>
def test_dataset_description_node(client):
UUID = "5b9ae1bd-e780-4869-a458-b3422084c480"
payload = {
"program": "demo1",
"project": "12L",
}
> response = client.post(
f"/record/{UUID}", json=payload)
tests/test_data.py:65:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:635: in post
return self.request("POST", url, data=data, json=json, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:476: in request
return super().request(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:587: in request
resp = self.send(prep, **send_kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:701: in send
r = adapter.send(request, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:270: in send
raise exc
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:267: in send
portal.call(self.app, scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:283: in call
return cast(T_Retval, self.start_task_soon(func, *args).result())
/usr/lib/python3.9/concurrent/futures/_base.py:446: in result
return self.__get_result()
/usr/lib/python3.9/concurrent/futures/_base.py:391: in __get_result
raise self._exception
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:219: in _call_func
retval = await retval
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/applications.py:269: in __call__
await super().__call__(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/applications.py:124: in __call__
await self.middleware_stack(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:184: in __call__
raise exc
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:162: in __call__
await self.app(scope, receive, _send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/cors.py:84: in __call__
await self.app(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:93: in __call__
raise exc
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:82: in __call__
await self.app(scope, receive, sender)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:21: in __call__
raise e
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:18: in __call__
await self.app(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:670: in __call__
await route.handle(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:266: in handle
await self.app(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:65: in app
response = await func(request)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:227: in app
raw_response = await run_endpoint_function(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:160: in run_endpoint_function
return await dependant.call(**values)
app/main.py:218: in get_gen3_record
record = SUBMISSION.export_record(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/submission.py:348: in export_record
output = requests.get(api_url, auth=self._auth_provider).text
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:73: in get
return request("get", url, params=params, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request
return session.request(method=method, url=url, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:573: in request
prep = self.prepare_request(req)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:484: in prepare_request
p.prepare(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:372: in prepare
self.prepare_auth(auth, url)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:603: in prepare_auth
r = auth(self)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:266: in __call__
request.headers["Authorization"] = self._get_auth_value()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:378: in _get_auth_value
return "bearer " + self.get_access_token()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:367: in get_access_token
return self.refresh_access_token(endpoint)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:304: in refresh_access_token
self._access_token = get_access_token_with_key(self._refresh_token)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:75: in get_access_token_with_key
resp = requests.post(auth_url, json=api_key)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:115: in post
return request("post", url, data=data, json=json, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request
return session.request(method=method, url=url, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:587: in request
resp = self.send(prep, **send_kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:701: in send
r = adapter.send(request, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <requests.adapters.HTTPAdapter object at 0x7f8b75a6c490>
request = <PreparedRequest [POST]>, stream = False
timeout = Timeout(connect=None, read=None, total=None), verify = True
cert = None, proxies = OrderedDict()
def send(
self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None
):
"""Sends PreparedRequest object. Returns Response object.
:param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
:param stream: (optional) Whether to stream the request content.
:param timeout: (optional) How long to wait for the server to send
data before giving up, as a float, or a :ref:`(connect timeout,
read timeout) <timeouts>` tuple.
:type timeout: float or tuple or urllib3 Timeout object
:param verify: (optional) Either a boolean, in which case it controls whether
we verify the server's TLS certificate, or a string, in which case it
must be a path to a CA bundle to use
:param cert: (optional) Any user-provided SSL certificate to be trusted.
:param proxies: (optional) The proxies dictionary to apply to the request.
:rtype: requests.Response
"""
try:
conn = self.get_connection(request.url, proxies)
except LocationValueError as e:
raise InvalidURL(e, request=request)
self.cert_verify(conn, request.url, verify, cert)
url = self.request_url(request, proxies)
self.add_headers(
request,
stream=stream,
timeout=timeout,
verify=verify,
cert=cert,
proxies=proxies,
)
chunked = not (request.body is None or "Content-Length" in request.headers)
if isinstance(timeout, tuple):
try:
connect, read = timeout
timeout = TimeoutSauce(connect=connect, read=read)
except ValueError:
raise ValueError(
f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, "
f"or a single float to set both timeouts to the same value."
)
elif isinstance(timeout, TimeoutSauce):
pass
else:
timeout = TimeoutSauce(connect=timeout, read=timeout)
try:
if not chunked:
resp = conn.urlopen(
method=request.method,
url=url,
body=request.body,
headers=request.headers,
redirect=False,
assert_same_host=False,
preload_content=False,
decode_content=False,
retries=self.max_retries,
timeout=timeout,
)
# Send the request.
else:
if hasattr(conn, "proxy_pool"):
conn = conn.proxy_pool
low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)
try:
skip_host = "Host" in request.headers
low_conn.putrequest(
request.method,
url,
skip_accept_encoding=True,
skip_host=skip_host,
)
for header, value in request.headers.items():
low_conn.putheader(header, value)
low_conn.endheaders()
for i in request.body:
low_conn.send(hex(len(i))[2:].encode("utf-8"))
low_conn.send(b"\r\n")
low_conn.send(i)
low_conn.send(b"\r\n")
low_conn.send(b"0\r\n\r\n")
# Receive the response from the server
r = low_conn.getresponse()
resp = HTTPResponse.from_httplib(
r,
pool=conn,
connection=low_conn,
preload_content=False,
decode_content=False,
)
except Exception:
# If we hit any problems here, clean up the connection.
# Then, raise so that we can handle the actual exception.
low_conn.close()
raise
except (ProtocolError, OSError) as err:
raise ConnectionError(err, request=request)
except MaxRetryError as e:
if isinstance(e.reason, ConnectTimeoutError):
# TODO: Remove this in 3.0.0: see #2811
if not isinstance(e.reason, NewConnectionError):
raise ConnectTimeout(e, request=request)
if isinstance(e.reason, ResponseError):
raise RetryError(e, request=request)
if isinstance(e.reason, _ProxyError):
raise ProxyError(e, request=request)
if isinstance(e.reason, _SSLError):
# This branch is for urllib3 v1.22 and later.
raise SSLError(e, request=request)
> raise ConnectionError(e, request=request)
E requests.exceptions.ConnectionError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8b75a6cac0>: Failed to establish a new connection: [Errno 110] Connection timed out'))
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:565: ConnectionError
______________________________ test_manifest_node ______________________________
self = <urllib3.connection.HTTPSConnection object at 0x7f8b75c61a90>
def _new_conn(self):
"""Establish a socket connection and set nodelay settings on it.
:return: New socket connection.
"""
extra_kw = {}
if self.source_address:
extra_kw["source_address"] = self.source_address
if self.socket_options:
extra_kw["socket_options"] = self.socket_options
try:
> conn = connection.create_connection(
(self._dns_host, self.port), self.timeout, **extra_kw
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:174:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
address = ('gen3.abi-ctt-ctp.cloud.edu.au', 443), timeout = None
source_address = None, socket_options = [(6, 1, 1)]
def create_connection(
address,
timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
source_address=None,
socket_options=None,
):
"""Connect to *address* and return the socket object.
Convenience function. Connect to *address* (a 2-tuple ``(host,
port)``) and return the socket object. Passing the optional
*timeout* parameter will set the timeout on the socket instance
before attempting to connect. If no *timeout* is supplied, the
global default timeout setting returned by :func:`socket.getdefaulttimeout`
is used. If *source_address* is set it must be a tuple of (host, port)
for the socket to bind as a source address before making the connection.
An host of '' or port 0 tells the OS to use the default.
"""
host, port = address
if host.startswith("["):
host = host.strip("[]")
err = None
# Using the value from allowed_gai_family() in the context of getaddrinfo lets
# us select whether to work with IPv4 DNS records, IPv6 records, or both.
# The original create_connection function always returns all records.
family = allowed_gai_family()
try:
host.encode("idna")
except UnicodeError:
return six.raise_from(
LocationParseError(u"'%s', label empty or too long" % host), None
)
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
af, socktype, proto, canonname, sa = res
sock = None
try:
sock = socket.socket(af, socktype, proto)
# If provided, set socket level options before connecting.
_set_socket_options(sock, socket_options)
if timeout is not socket._GLOBAL_DEFAULT_TIMEOUT:
sock.settimeout(timeout)
if source_address:
sock.bind(source_address)
sock.connect(sa)
return sock
except socket.error as e:
err = e
if sock is not None:
sock.close()
sock = None
if err is not None:
> raise err
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/connection.py:95:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
address = ('gen3.abi-ctt-ctp.cloud.edu.au', 443), timeout = None
source_address = None, socket_options = [(6, 1, 1)]
def create_connection(
address,
timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
source_address=None,
socket_options=None,
):
"""Connect to *address* and return the socket object.
Convenience function. Connect to *address* (a 2-tuple ``(host,
port)``) and return the socket object. Passing the optional
*timeout* parameter will set the timeout on the socket instance
before attempting to connect. If no *timeout* is supplied, the
global default timeout setting returned by :func:`socket.getdefaulttimeout`
is used. If *source_address* is set it must be a tuple of (host, port)
for the socket to bind as a source address before making the connection.
An host of '' or port 0 tells the OS to use the default.
"""
host, port = address
if host.startswith("["):
host = host.strip("[]")
err = None
# Using the value from allowed_gai_family() in the context of getaddrinfo lets
# us select whether to work with IPv4 DNS records, IPv6 records, or both.
# The original create_connection function always returns all records.
family = allowed_gai_family()
try:
host.encode("idna")
except UnicodeError:
return six.raise_from(
LocationParseError(u"'%s', label empty or too long" % host), None
)
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
af, socktype, proto, canonname, sa = res
sock = None
try:
sock = socket.socket(af, socktype, proto)
# If provided, set socket level options before connecting.
_set_socket_options(sock, socket_options)
if timeout is not socket._GLOBAL_DEFAULT_TIMEOUT:
sock.settimeout(timeout)
if source_address:
sock.bind(source_address)
> sock.connect(sa)
E TimeoutError: [Errno 110] Connection timed out
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/connection.py:85: TimeoutError
During handling of the above exception, another exception occurred:
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b75d30c40>
method = 'POST', url = '/user/credentials/cdis/access_token'
body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...vPrcq8DT5FB92JLhLStE4nkB9Ysm96h77xFJ44rGq3TODjZaY6Tdj1JgHQrf8BoV2Q", "key_id": "cc7139d4-a9a2-45db-98f8-309087b6b674"}'
headers = {'User-Agent': 'python-requests/2.28.2', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}
retries = Retry(total=0, connect=None, read=False, redirect=None, status=None)
redirect = False, assert_same_host = False
timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None
release_conn = False, chunked = False, body_pos = None
response_kw = {'decode_content': False, 'preload_content': False}
parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None)
destination_scheme = None, conn = None, release_this_conn = True
http_tunnel_required = False, err = None, clean_exit = False
def urlopen(
self,
method,
url,
body=None,
headers=None,
retries=None,
redirect=True,
assert_same_host=True,
timeout=_Default,
pool_timeout=None,
release_conn=None,
chunked=False,
body_pos=None,
**response_kw
):
"""
Get a connection from the pool and perform an HTTP request. This is the
lowest level call for making a request, so you'll need to specify all
the raw details.
.. note::
More commonly, it's appropriate to use a convenience method provided
by :class:`.RequestMethods`, such as :meth:`request`.
.. note::
`release_conn` will only behave as expected if
`preload_content=False` because we want to make
`preload_content=False` the default behaviour someday soon without
breaking backwards compatibility.
:param method:
HTTP request method (such as GET, POST, PUT, etc.)
:param url:
The URL to perform the request on.
:param body:
Data to send in the request body, either :class:`str`, :class:`bytes`,
an iterable of :class:`str`/:class:`bytes`, or a file-like object.
:param headers:
Dictionary of custom headers to send, such as User-Agent,
If-None-Match, etc. If None, pool headers are used. If provided,
these headers completely replace any pool-specific headers.
:param retries:
Configure the number of retries to allow before raising a
:class:`~urllib3.exceptions.MaxRetryError` exception.
Pass ``None`` to retry until you receive a response. Pass a
:class:`~urllib3.util.retry.Retry` object for fine-grained control
over different types of retries.
Pass an integer number to retry connection errors that many times,
but no other types of errors. Pass zero to never retry.
If ``False``, then retries are disabled and any exception is raised
immediately. Also, instead of raising a MaxRetryError on redirects,
the redirect response will be returned.
:type retries: :class:`~urllib3.util.retry.Retry`, False, or an int.
:param redirect:
If True, automatically handle redirects (status codes 301, 302,
303, 307, 308). Each redirect counts as a retry. Disabling retries
will disable redirect, too.
:param assert_same_host:
If ``True``, will make sure that the host of the pool requests is
consistent else will raise HostChangedError. When ``False``, you can
use the pool on an HTTP proxy and request foreign hosts.
:param timeout:
If specified, overrides the default timeout for this one
request. It may be a float (in seconds) or an instance of
:class:`urllib3.util.Timeout`.
:param pool_timeout:
If set and the pool is set to block=True, then this method will
block for ``pool_timeout`` seconds and raise EmptyPoolError if no
connection is available within the time period.
:param release_conn:
If False, then the urlopen call will not release the connection
back into the pool once a response is received (but will release if
you read the entire contents of the response such as when
`preload_content=True`). This is useful if you're not preloading
the response's content immediately. You will need to call
``r.release_conn()`` on the response ``r`` to return the connection
back into the pool. If None, it takes the value of
``response_kw.get('preload_content', True)``.
:param chunked:
If True, urllib3 will send the body using chunked transfer
encoding. Otherwise, urllib3 will send the body using the standard
content-length form. Defaults to False.
:param int body_pos:
Position to seek to in file-like body in the event of a retry or
redirect. Typically this won't need to be set because urllib3 will
auto-populate the value when needed.
:param \\**response_kw:
Additional parameters are passed to
:meth:`urllib3.response.HTTPResponse.from_httplib`
"""
parsed_url = parse_url(url)
destination_scheme = parsed_url.scheme
if headers is None:
headers = self.headers
if not isinstance(retries, Retry):
retries = Retry.from_int(retries, redirect=redirect, default=self.retries)
if release_conn is None:
release_conn = response_kw.get("preload_content", True)
# Check host
if assert_same_host and not self.is_same_host(url):
raise HostChangedError(self, url, retries)
# Ensure that the URL we're connecting to is properly encoded
if url.startswith("/"):
url = six.ensure_str(_encode_target(url))
else:
url = six.ensure_str(parsed_url.url)
conn = None
# Track whether `conn` needs to be released before
# returning/raising/recursing. Update this variable if necessary, and
# leave `release_conn` constant throughout the function. That way, if
# the function recurses, the original value of `release_conn` will be
# passed down into the recursive call, and its value will be respected.
#
# See issue #651 [1] for details.
#
# [1] <https://github.com/urllib3/urllib3/issues/651>
release_this_conn = release_conn
http_tunnel_required = connection_requires_http_tunnel(
self.proxy, self.proxy_config, destination_scheme
)
# Merge the proxy headers. Only done when not using HTTP CONNECT. We
# have to copy the headers dict so we can safely change it without those
# changes being reflected in anyone else's copy.
if not http_tunnel_required:
headers = headers.copy()
headers.update(self.proxy_headers)
# Must keep the exception bound to a separate variable or else Python 3
# complains about UnboundLocalError.
err = None
# Keep track of whether we cleanly exited the except block. This
# ensures we do proper cleanup in finally.
clean_exit = False
# Rewind body position, if needed. Record current position
# for future rewinds in the event of a redirect/retry.
body_pos = set_file_position(body, body_pos)
try:
# Request a connection from the queue.
timeout_obj = self._get_timeout(timeout)
conn = self._get_conn(timeout=pool_timeout)
conn.timeout = timeout_obj.connect_timeout
is_new_proxy_conn = self.proxy is not None and not getattr(
conn, "sock", None
)
if is_new_proxy_conn and http_tunnel_required:
self._prepare_proxy(conn)
# Make the request on the httplib connection object.
> httplib_response = self._make_request(
conn,
method,
url,
timeout=timeout_obj,
body=body,
headers=headers,
chunked=chunked,
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:703:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b75d30c40>
conn = <urllib3.connection.HTTPSConnection object at 0x7f8b75c61a90>
method = 'POST', url = '/user/credentials/cdis/access_token'
timeout = Timeout(connect=None, read=None, total=None), chunked = False
httplib_request_kw = {'body': b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwd...p, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}}
timeout_obj = Timeout(connect=None, read=None, total=None)
def _make_request(
self, conn, method, url, timeout=_Default, chunked=False, **httplib_request_kw
):
"""
Perform a request on a given urllib connection object taken from our
pool.
:param conn:
a connection from one of our connection pools
:param timeout:
Socket timeout in seconds for the request. This can be a
float or integer, which will set the same timeout value for
the socket connect and the socket read, or an instance of
:class:`urllib3.util.Timeout`, which gives you more fine-grained
control over your timeouts.
"""
self.num_requests += 1
timeout_obj = self._get_timeout(timeout)
timeout_obj.start_connect()
conn.timeout = timeout_obj.connect_timeout
# Trigger any extra validation we need to do.
try:
> self._validate_conn(conn)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:386:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b75d30c40>
conn = <urllib3.connection.HTTPSConnection object at 0x7f8b75c61a90>
def _validate_conn(self, conn):
"""
Called right before a request is made, after the socket is created.
"""
super(HTTPSConnectionPool, self)._validate_conn(conn)
# Force connect early to allow us to validate the connection.
if not getattr(conn, "sock", None): # AppEngine might not have `.sock`
> conn.connect()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:1042:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connection.HTTPSConnection object at 0x7f8b75c61a90>
def connect(self):
# Add certificate verification
> self.sock = conn = self._new_conn()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:358:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connection.HTTPSConnection object at 0x7f8b75c61a90>
def _new_conn(self):
"""Establish a socket connection and set nodelay settings on it.
:return: New socket connection.
"""
extra_kw = {}
if self.source_address:
extra_kw["source_address"] = self.source_address
if self.socket_options:
extra_kw["socket_options"] = self.socket_options
try:
conn = connection.create_connection(
(self._dns_host, self.port), self.timeout, **extra_kw
)
except SocketTimeout:
raise ConnectTimeoutError(
self,
"Connection to %s timed out. (connect timeout=%s)"
% (self.host, self.timeout),
)
except SocketError as e:
> raise NewConnectionError(
self, "Failed to establish a new connection: %s" % e
)
E urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f8b75c61a90>: Failed to establish a new connection: [Errno 110] Connection timed out
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:186: NewConnectionError
During handling of the above exception, another exception occurred:
self = <requests.adapters.HTTPAdapter object at 0x7f8b75d30940>
request = <PreparedRequest [POST]>, stream = False
timeout = Timeout(connect=None, read=None, total=None), verify = True
cert = None, proxies = OrderedDict()
def send(
self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None
):
"""Sends PreparedRequest object. Returns Response object.
:param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
:param stream: (optional) Whether to stream the request content.
:param timeout: (optional) How long to wait for the server to send
data before giving up, as a float, or a :ref:`(connect timeout,
read timeout) <timeouts>` tuple.
:type timeout: float or tuple or urllib3 Timeout object
:param verify: (optional) Either a boolean, in which case it controls whether
we verify the server's TLS certificate, or a string, in which case it
must be a path to a CA bundle to use
:param cert: (optional) Any user-provided SSL certificate to be trusted.
:param proxies: (optional) The proxies dictionary to apply to the request.
:rtype: requests.Response
"""
try:
conn = self.get_connection(request.url, proxies)
except LocationValueError as e:
raise InvalidURL(e, request=request)
self.cert_verify(conn, request.url, verify, cert)
url = self.request_url(request, proxies)
self.add_headers(
request,
stream=stream,
timeout=timeout,
verify=verify,
cert=cert,
proxies=proxies,
)
chunked = not (request.body is None or "Content-Length" in request.headers)
if isinstance(timeout, tuple):
try:
connect, read = timeout
timeout = TimeoutSauce(connect=connect, read=read)
except ValueError:
raise ValueError(
f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, "
f"or a single float to set both timeouts to the same value."
)
elif isinstance(timeout, TimeoutSauce):
pass
else:
timeout = TimeoutSauce(connect=timeout, read=timeout)
try:
if not chunked:
> resp = conn.urlopen(
method=request.method,
url=url,
body=request.body,
headers=request.headers,
redirect=False,
assert_same_host=False,
preload_content=False,
decode_content=False,
retries=self.max_retries,
timeout=timeout,
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:489:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b75d30c40>
method = 'POST', url = '/user/credentials/cdis/access_token'
body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...vPrcq8DT5FB92JLhLStE4nkB9Ysm96h77xFJ44rGq3TODjZaY6Tdj1JgHQrf8BoV2Q", "key_id": "cc7139d4-a9a2-45db-98f8-309087b6b674"}'
headers = {'User-Agent': 'python-requests/2.28.2', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}
retries = Retry(total=0, connect=None, read=False, redirect=None, status=None)
redirect = False, assert_same_host = False
timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None
release_conn = False, chunked = False, body_pos = None
response_kw = {'decode_content': False, 'preload_content': False}
parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None)
destination_scheme = None, conn = None, release_this_conn = True
http_tunnel_required = False, err = None, clean_exit = False
def urlopen(
self,
method,
url,
body=None,
headers=None,
retries=None,
redirect=True,
assert_same_host=True,
timeout=_Default,
pool_timeout=None,
release_conn=None,
chunked=False,
body_pos=None,
**response_kw
):
"""
Get a connection from the pool and perform an HTTP request. This is the
lowest level call for making a request, so you'll need to specify all
the raw details.
.. note::
More commonly, it's appropriate to use a convenience method provided
by :class:`.RequestMethods`, such as :meth:`request`.
.. note::
`release_conn` will only behave as expected if
`preload_content=False` because we want to make
`preload_content=False` the default behaviour someday soon without
breaking backwards compatibility.
:param method:
HTTP request method (such as GET, POST, PUT, etc.)
:param url:
The URL to perform the request on.
:param body:
Data to send in the request body, either :class:`str`, :class:`bytes`,
an iterable of :class:`str`/:class:`bytes`, or a file-like object.
:param headers:
Dictionary of custom headers to send, such as User-Agent,
If-None-Match, etc. If None, pool headers are used. If provided,
these headers completely replace any pool-specific headers.
:param retries:
Configure the number of retries to allow before raising a
:class:`~urllib3.exceptions.MaxRetryError` exception.
Pass ``None`` to retry until you receive a response. Pass a
:class:`~urllib3.util.retry.Retry` object for fine-grained control
over different types of retries.
Pass an integer number to retry connection errors that many times,
but no other types of errors. Pass zero to never retry.
If ``False``, then retries are disabled and any exception is raised
immediately. Also, instead of raising a MaxRetryError on redirects,
the redirect response will be returned.
:type retries: :class:`~urllib3.util.retry.Retry`, False, or an int.
:param redirect:
If True, automatically handle redirects (status codes 301, 302,
303, 307, 308). Each redirect counts as a retry. Disabling retries
will disable redirect, too.
:param assert_same_host:
If ``True``, will make sure that the host of the pool requests is
consistent else will raise HostChangedError. When ``False``, you can
use the pool on an HTTP proxy and request foreign hosts.
:param timeout:
If specified, overrides the default timeout for this one
request. It may be a float (in seconds) or an instance of
:class:`urllib3.util.Timeout`.
:param pool_timeout:
If set and the pool is set to block=True, then this method will
block for ``pool_timeout`` seconds and raise EmptyPoolError if no
connection is available within the time period.
:param release_conn:
If False, then the urlopen call will not release the connection
back into the pool once a response is received (but will release if
you read the entire contents of the response such as when
`preload_content=True`). This is useful if you're not preloading
the response's content immediately. You will need to call
``r.release_conn()`` on the response ``r`` to return the connection
back into the pool. If None, it takes the value of
``response_kw.get('preload_content', True)``.
:param chunked:
If True, urllib3 will send the body using chunked transfer
encoding. Otherwise, urllib3 will send the body using the standard
content-length form. Defaults to False.
:param int body_pos:
Position to seek to in file-like body in the event of a retry or
redirect. Typically this won't need to be set because urllib3 will
auto-populate the value when needed.
:param \\**response_kw:
Additional parameters are passed to
:meth:`urllib3.response.HTTPResponse.from_httplib`
"""
parsed_url = parse_url(url)
destination_scheme = parsed_url.scheme
if headers is None:
headers = self.headers
if not isinstance(retries, Retry):
retries = Retry.from_int(retries, redirect=redirect, default=self.retries)
if release_conn is None:
release_conn = response_kw.get("preload_content", True)
# Check host
if assert_same_host and not self.is_same_host(url):
raise HostChangedError(self, url, retries)
# Ensure that the URL we're connecting to is properly encoded
if url.startswith("/"):
url = six.ensure_str(_encode_target(url))
else:
url = six.ensure_str(parsed_url.url)
conn = None
# Track whether `conn` needs to be released before
# returning/raising/recursing. Update this variable if necessary, and
# leave `release_conn` constant throughout the function. That way, if
# the function recurses, the original value of `release_conn` will be
# passed down into the recursive call, and its value will be respected.
#
# See issue #651 [1] for details.
#
# [1] <https://github.com/urllib3/urllib3/issues/651>
release_this_conn = release_conn
http_tunnel_required = connection_requires_http_tunnel(
self.proxy, self.proxy_config, destination_scheme
)
# Merge the proxy headers. Only done when not using HTTP CONNECT. We
# have to copy the headers dict so we can safely change it without those
# changes being reflected in anyone else's copy.
if not http_tunnel_required:
headers = headers.copy()
headers.update(self.proxy_headers)
# Must keep the exception bound to a separate variable or else Python 3
# complains about UnboundLocalError.
err = None
# Keep track of whether we cleanly exited the except block. This
# ensures we do proper cleanup in finally.
clean_exit = False
# Rewind body position, if needed. Record current position
# for future rewinds in the event of a redirect/retry.
body_pos = set_file_position(body, body_pos)
try:
# Request a connection from the queue.
timeout_obj = self._get_timeout(timeout)
conn = self._get_conn(timeout=pool_timeout)
conn.timeout = timeout_obj.connect_timeout
is_new_proxy_conn = self.proxy is not None and not getattr(
conn, "sock", None
)
if is_new_proxy_conn and http_tunnel_required:
self._prepare_proxy(conn)
# Make the request on the httplib connection object.
httplib_response = self._make_request(
conn,
method,
url,
timeout=timeout_obj,
body=body,
headers=headers,
chunked=chunked,
)
# If we're going to release the connection in ``finally:``, then
# the response doesn't need to know about the connection. Otherwise
# it will also try to release it and we'll have a double-release
# mess.
response_conn = conn if not release_conn else None
# Pass method to Response for length checking
response_kw["request_method"] = method
# Import httplib's response into our own wrapper object
response = self.ResponseCls.from_httplib(
httplib_response,
pool=self,
connection=response_conn,
retries=retries,
**response_kw
)
# Everything went great!
clean_exit = True
except EmptyPoolError:
# Didn't get a connection from the pool, no need to clean up
clean_exit = True
release_this_conn = False
raise
except (
TimeoutError,
HTTPException,
SocketError,
ProtocolError,
BaseSSLError,
SSLError,
CertificateError,
) as e:
# Discard the connection for these exceptions. It will be
# replaced during the next _get_conn() call.
clean_exit = False
def _is_ssl_error_message_from_http_proxy(ssl_error):
# We're trying to detect the message 'WRONG_VERSION_NUMBER' but
# SSLErrors are kinda all over the place when it comes to the message,
# so we try to cover our bases here!
message = " ".join(re.split("[^a-z]", str(ssl_error).lower()))
return (
"wrong version number" in message or "unknown protocol" in message
)
# Try to detect a common user error with proxies which is to
# set an HTTP proxy to be HTTPS when it should be 'http://'
# (ie {'http': 'http://proxy', 'https': 'https://proxy'})
# Instead we add a nice error message and point to a URL.
if (
isinstance(e, BaseSSLError)
and self.proxy
and _is_ssl_error_message_from_http_proxy(e)
and conn.proxy
and conn.proxy.scheme == "https"
):
e = ProxyError(
"Your proxy appears to only use HTTP and not HTTPS, "
"try changing your proxy URL to be HTTP. See: "
"https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html"
"#https-proxy-error-http-proxy",
SSLError(e),
)
elif isinstance(e, (BaseSSLError, CertificateError)):
e = SSLError(e)
elif isinstance(e, (SocketError, NewConnectionError)) and self.proxy:
e = ProxyError("Cannot connect to proxy.", e)
elif isinstance(e, (SocketError, HTTPException)):
e = ProtocolError("Connection aborted.", e)
> retries = retries.increment(
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:787:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = Retry(total=0, connect=None, read=False, redirect=None, status=None)
method = 'POST', url = '/user/credentials/cdis/access_token', response = None
error = NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8b75c61a90>: Failed to establish a new connection: [Errno 110] Connection timed out')
_pool = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b75d30c40>
_stacktrace = <traceback object at 0x7f8b75d5a680>
def increment(
self,
method=None,
url=None,
response=None,
error=None,
_pool=None,
_stacktrace=None,
):
"""Return a new Retry object with incremented retry counters.
:param response: A response object, or None, if the server did not
return a response.
:type response: :class:`~urllib3.response.HTTPResponse`
:param Exception error: An error encountered during the request, or
None if the response was received successfully.
:return: A new ``Retry`` object.
"""
if self.total is False and error:
# Disabled, indicate to re-raise the error.
raise six.reraise(type(error), error, _stacktrace)
total = self.total
if total is not None:
total -= 1
connect = self.connect
read = self.read
redirect = self.redirect
status_count = self.status
other = self.other
cause = "unknown"
status = None
redirect_location = None
if error and self._is_connection_error(error):
# Connect retry?
if connect is False:
raise six.reraise(type(error), error, _stacktrace)
elif connect is not None:
connect -= 1
elif error and self._is_read_error(error):
# Read retry?
if read is False or not self._is_method_retryable(method):
raise six.reraise(type(error), error, _stacktrace)
elif read is not None:
read -= 1
elif error:
# Other retry?
if other is not None:
other -= 1
elif response and response.get_redirect_location():
# Redirect retry?
if redirect is not None:
redirect -= 1
cause = "too many redirects"
redirect_location = response.get_redirect_location()
status = response.status
else:
# Incrementing because of a server error like a 500 in
# status_forcelist and the given method is in the allowed_methods
cause = ResponseError.GENERIC_ERROR
if response and response.status:
if status_count is not None:
status_count -= 1
cause = ResponseError.SPECIFIC_ERROR.format(status_code=response.status)
status = response.status
history = self.history + (
RequestHistory(method, url, error, status, redirect_location),
)
new_retry = self.new(
total=total,
connect=connect,
read=read,
redirect=redirect,
status=status_count,
other=other,
history=history,
)
if new_retry.is_exhausted():
> raise MaxRetryError(_pool, url, error or ResponseError(cause))
E urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8b75c61a90>: Failed to establish a new connection: [Errno 110] Connection timed out'))
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/retry.py:592: MaxRetryError
During handling of the above exception, another exception occurred:
client = <starlette.testclient.TestClient object at 0x7f8b7ee5e9a0>
def test_manifest_node(client):
UUID = "fd65a93f-ff62-45e4-b7b6-96419ef4f749"
payload = {
"program": "demo1",
"project": "12L",
}
> response = client.post(
f"/record/{UUID}", json=payload)
tests/test_data.py:169:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:635: in post
return self.request("POST", url, data=data, json=json, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:476: in request
return super().request(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:587: in request
resp = self.send(prep, **send_kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:701: in send
r = adapter.send(request, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:270: in send
raise exc
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:267: in send
portal.call(self.app, scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:283: in call
return cast(T_Retval, self.start_task_soon(func, *args).result())
/usr/lib/python3.9/concurrent/futures/_base.py:446: in result
return self.__get_result()
/usr/lib/python3.9/concurrent/futures/_base.py:391: in __get_result
raise self._exception
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:219: in _call_func
retval = await retval
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/applications.py:269: in __call__
await super().__call__(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/applications.py:124: in __call__
await self.middleware_stack(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:184: in __call__
raise exc
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:162: in __call__
await self.app(scope, receive, _send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/cors.py:84: in __call__
await self.app(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:93: in __call__
raise exc
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:82: in __call__
await self.app(scope, receive, sender)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:21: in __call__
raise e
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:18: in __call__
await self.app(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:670: in __call__
await route.handle(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:266: in handle
await self.app(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:65: in app
response = await func(request)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:227: in app
raw_response = await run_endpoint_function(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:160: in run_endpoint_function
return await dependant.call(**values)
app/main.py:218: in get_gen3_record
record = SUBMISSION.export_record(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/submission.py:348: in export_record
output = requests.get(api_url, auth=self._auth_provider).text
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:73: in get
return request("get", url, params=params, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request
return session.request(method=method, url=url, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:573: in request
prep = self.prepare_request(req)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:484: in prepare_request
p.prepare(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:372: in prepare
self.prepare_auth(auth, url)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:603: in prepare_auth
r = auth(self)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:266: in __call__
request.headers["Authorization"] = self._get_auth_value()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:378: in _get_auth_value
return "bearer " + self.get_access_token()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:367: in get_access_token
return self.refresh_access_token(endpoint)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:304: in refresh_access_token
self._access_token = get_access_token_with_key(self._refresh_token)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:75: in get_access_token_with_key
resp = requests.post(auth_url, json=api_key)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:115: in post
return request("post", url, data=data, json=json, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request
return session.request(method=method, url=url, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:587: in request
resp = self.send(prep, **send_kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:701: in send
r = adapter.send(request, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <requests.adapters.HTTPAdapter object at 0x7f8b75d30940>
request = <PreparedRequest [POST]>, stream = False
timeout = Timeout(connect=None, read=None, total=None), verify = True
cert = None, proxies = OrderedDict()
def send(
self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None
):
"""Sends PreparedRequest object. Returns Response object.
:param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
:param stream: (optional) Whether to stream the request content.
:param timeout: (optional) How long to wait for the server to send
data before giving up, as a float, or a :ref:`(connect timeout,
read timeout) <timeouts>` tuple.
:type timeout: float or tuple or urllib3 Timeout object
:param verify: (optional) Either a boolean, in which case it controls whether
we verify the server's TLS certificate, or a string, in which case it
must be a path to a CA bundle to use
:param cert: (optional) Any user-provided SSL certificate to be trusted.
:param proxies: (optional) The proxies dictionary to apply to the request.
:rtype: requests.Response
"""
try:
conn = self.get_connection(request.url, proxies)
except LocationValueError as e:
raise InvalidURL(e, request=request)
self.cert_verify(conn, request.url, verify, cert)
url = self.request_url(request, proxies)
self.add_headers(
request,
stream=stream,
timeout=timeout,
verify=verify,
cert=cert,
proxies=proxies,
)
chunked = not (request.body is None or "Content-Length" in request.headers)
if isinstance(timeout, tuple):
try:
connect, read = timeout
timeout = TimeoutSauce(connect=connect, read=read)
except ValueError:
raise ValueError(
f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, "
f"or a single float to set both timeouts to the same value."
)
elif isinstance(timeout, TimeoutSauce):
pass
else:
timeout = TimeoutSauce(connect=timeout, read=timeout)
try:
if not chunked:
resp = conn.urlopen(
method=request.method,
url=url,
body=request.body,
headers=request.headers,
redirect=False,
assert_same_host=False,
preload_content=False,
decode_content=False,
retries=self.max_retries,
timeout=timeout,
)
# Send the request.
else:
if hasattr(conn, "proxy_pool"):
conn = conn.proxy_pool
low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)
try:
skip_host = "Host" in request.headers
low_conn.putrequest(
request.method,
url,
skip_accept_encoding=True,
skip_host=skip_host,
)
for header, value in request.headers.items():
low_conn.putheader(header, value)
low_conn.endheaders()
for i in request.body:
low_conn.send(hex(len(i))[2:].encode("utf-8"))
low_conn.send(b"\r\n")
low_conn.send(i)
low_conn.send(b"\r\n")
low_conn.send(b"0\r\n\r\n")
# Receive the response from the server
r = low_conn.getresponse()
resp = HTTPResponse.from_httplib(
r,
pool=conn,
connection=low_conn,
preload_content=False,
decode_content=False,
)
except Exception:
# If we hit any problems here, clean up the connection.
# Then, raise so that we can handle the actual exception.
low_conn.close()
raise
except (ProtocolError, OSError) as err:
raise ConnectionError(err, request=request)
except MaxRetryError as e:
if isinstance(e.reason, ConnectTimeoutError):
# TODO: Remove this in 3.0.0: see #2811
if not isinstance(e.reason, NewConnectionError):
raise ConnectTimeout(e, request=request)
if isinstance(e.reason, ResponseError):
raise RetryError(e, request=request)
if isinstance(e.reason, _ProxyError):
raise ProxyError(e, request=request)
if isinstance(e.reason, _SSLError):
# This branch is for urllib3 v1.22 and later.
raise SSLError(e, request=request)
> raise ConnectionError(e, request=request)
E requests.exceptions.ConnectionError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8b75c61a90>: Failed to establish a new connection: [Errno 110] Connection timed out'))
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:565: ConnectionError
________________________________ test_case_node ________________________________
self = <urllib3.connection.HTTPSConnection object at 0x7f8b76022b50>
def _new_conn(self):
"""Establish a socket connection and set nodelay settings on it.
:return: New socket connection.
"""
extra_kw = {}
if self.source_address:
extra_kw["source_address"] = self.source_address
if self.socket_options:
extra_kw["socket_options"] = self.socket_options
try:
> conn = connection.create_connection(
(self._dns_host, self.port), self.timeout, **extra_kw
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:174:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
address = ('gen3.abi-ctt-ctp.cloud.edu.au', 443), timeout = None
source_address = None, socket_options = [(6, 1, 1)]
def create_connection(
address,
timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
source_address=None,
socket_options=None,
):
"""Connect to *address* and return the socket object.
Convenience function. Connect to *address* (a 2-tuple ``(host,
port)``) and return the socket object. Passing the optional
*timeout* parameter will set the timeout on the socket instance
before attempting to connect. If no *timeout* is supplied, the
global default timeout setting returned by :func:`socket.getdefaulttimeout`
is used. If *source_address* is set it must be a tuple of (host, port)
for the socket to bind as a source address before making the connection.
An host of '' or port 0 tells the OS to use the default.
"""
host, port = address
if host.startswith("["):
host = host.strip("[]")
err = None
# Using the value from allowed_gai_family() in the context of getaddrinfo lets
# us select whether to work with IPv4 DNS records, IPv6 records, or both.
# The original create_connection function always returns all records.
family = allowed_gai_family()
try:
host.encode("idna")
except UnicodeError:
return six.raise_from(
LocationParseError(u"'%s', label empty or too long" % host), None
)
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
af, socktype, proto, canonname, sa = res
sock = None
try:
sock = socket.socket(af, socktype, proto)
# If provided, set socket level options before connecting.
_set_socket_options(sock, socket_options)
if timeout is not socket._GLOBAL_DEFAULT_TIMEOUT:
sock.settimeout(timeout)
if source_address:
sock.bind(source_address)
sock.connect(sa)
return sock
except socket.error as e:
err = e
if sock is not None:
sock.close()
sock = None
if err is not None:
> raise err
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/connection.py:95:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
address = ('gen3.abi-ctt-ctp.cloud.edu.au', 443), timeout = None
source_address = None, socket_options = [(6, 1, 1)]
def create_connection(
address,
timeout=socket._GLOBAL_DEFAULT_TIMEOUT,
source_address=None,
socket_options=None,
):
"""Connect to *address* and return the socket object.
Convenience function. Connect to *address* (a 2-tuple ``(host,
port)``) and return the socket object. Passing the optional
*timeout* parameter will set the timeout on the socket instance
before attempting to connect. If no *timeout* is supplied, the
global default timeout setting returned by :func:`socket.getdefaulttimeout`
is used. If *source_address* is set it must be a tuple of (host, port)
for the socket to bind as a source address before making the connection.
An host of '' or port 0 tells the OS to use the default.
"""
host, port = address
if host.startswith("["):
host = host.strip("[]")
err = None
# Using the value from allowed_gai_family() in the context of getaddrinfo lets
# us select whether to work with IPv4 DNS records, IPv6 records, or both.
# The original create_connection function always returns all records.
family = allowed_gai_family()
try:
host.encode("idna")
except UnicodeError:
return six.raise_from(
LocationParseError(u"'%s', label empty or too long" % host), None
)
for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):
af, socktype, proto, canonname, sa = res
sock = None
try:
sock = socket.socket(af, socktype, proto)
# If provided, set socket level options before connecting.
_set_socket_options(sock, socket_options)
if timeout is not socket._GLOBAL_DEFAULT_TIMEOUT:
sock.settimeout(timeout)
if source_address:
sock.bind(source_address)
> sock.connect(sa)
E TimeoutError: [Errno 110] Connection timed out
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/connection.py:85: TimeoutError
During handling of the above exception, another exception occurred:
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b75ad3280>
method = 'POST', url = '/user/credentials/cdis/access_token'
body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...vPrcq8DT5FB92JLhLStE4nkB9Ysm96h77xFJ44rGq3TODjZaY6Tdj1JgHQrf8BoV2Q", "key_id": "cc7139d4-a9a2-45db-98f8-309087b6b674"}'
headers = {'User-Agent': 'python-requests/2.28.2', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}
retries = Retry(total=0, connect=None, read=False, redirect=None, status=None)
redirect = False, assert_same_host = False
timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None
release_conn = False, chunked = False, body_pos = None
response_kw = {'decode_content': False, 'preload_content': False}
parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None)
destination_scheme = None, conn = None, release_this_conn = True
http_tunnel_required = False, err = None, clean_exit = False
def urlopen(
self,
method,
url,
body=None,
headers=None,
retries=None,
redirect=True,
assert_same_host=True,
timeout=_Default,
pool_timeout=None,
release_conn=None,
chunked=False,
body_pos=None,
**response_kw
):
"""
Get a connection from the pool and perform an HTTP request. This is the
lowest level call for making a request, so you'll need to specify all
the raw details.
.. note::
More commonly, it's appropriate to use a convenience method provided
by :class:`.RequestMethods`, such as :meth:`request`.
.. note::
`release_conn` will only behave as expected if
`preload_content=False` because we want to make
`preload_content=False` the default behaviour someday soon without
breaking backwards compatibility.
:param method:
HTTP request method (such as GET, POST, PUT, etc.)
:param url:
The URL to perform the request on.
:param body:
Data to send in the request body, either :class:`str`, :class:`bytes`,
an iterable of :class:`str`/:class:`bytes`, or a file-like object.
:param headers:
Dictionary of custom headers to send, such as User-Agent,
If-None-Match, etc. If None, pool headers are used. If provided,
these headers completely replace any pool-specific headers.
:param retries:
Configure the number of retries to allow before raising a
:class:`~urllib3.exceptions.MaxRetryError` exception.
Pass ``None`` to retry until you receive a response. Pass a
:class:`~urllib3.util.retry.Retry` object for fine-grained control
over different types of retries.
Pass an integer number to retry connection errors that many times,
but no other types of errors. Pass zero to never retry.
If ``False``, then retries are disabled and any exception is raised
immediately. Also, instead of raising a MaxRetryError on redirects,
the redirect response will be returned.
:type retries: :class:`~urllib3.util.retry.Retry`, False, or an int.
:param redirect:
If True, automatically handle redirects (status codes 301, 302,
303, 307, 308). Each redirect counts as a retry. Disabling retries
will disable redirect, too.
:param assert_same_host:
If ``True``, will make sure that the host of the pool requests is
consistent else will raise HostChangedError. When ``False``, you can
use the pool on an HTTP proxy and request foreign hosts.
:param timeout:
If specified, overrides the default timeout for this one
request. It may be a float (in seconds) or an instance of
:class:`urllib3.util.Timeout`.
:param pool_timeout:
If set and the pool is set to block=True, then this method will
block for ``pool_timeout`` seconds and raise EmptyPoolError if no
connection is available within the time period.
:param release_conn:
If False, then the urlopen call will not release the connection
back into the pool once a response is received (but will release if
you read the entire contents of the response such as when
`preload_content=True`). This is useful if you're not preloading
the response's content immediately. You will need to call
``r.release_conn()`` on the response ``r`` to return the connection
back into the pool. If None, it takes the value of
``response_kw.get('preload_content', True)``.
:param chunked:
If True, urllib3 will send the body using chunked transfer
encoding. Otherwise, urllib3 will send the body using the standard
content-length form. Defaults to False.
:param int body_pos:
Position to seek to in file-like body in the event of a retry or
redirect. Typically this won't need to be set because urllib3 will
auto-populate the value when needed.
:param \\**response_kw:
Additional parameters are passed to
:meth:`urllib3.response.HTTPResponse.from_httplib`
"""
parsed_url = parse_url(url)
destination_scheme = parsed_url.scheme
if headers is None:
headers = self.headers
if not isinstance(retries, Retry):
retries = Retry.from_int(retries, redirect=redirect, default=self.retries)
if release_conn is None:
release_conn = response_kw.get("preload_content", True)
# Check host
if assert_same_host and not self.is_same_host(url):
raise HostChangedError(self, url, retries)
# Ensure that the URL we're connecting to is properly encoded
if url.startswith("/"):
url = six.ensure_str(_encode_target(url))
else:
url = six.ensure_str(parsed_url.url)
conn = None
# Track whether `conn` needs to be released before
# returning/raising/recursing. Update this variable if necessary, and
# leave `release_conn` constant throughout the function. That way, if
# the function recurses, the original value of `release_conn` will be
# passed down into the recursive call, and its value will be respected.
#
# See issue #651 [1] for details.
#
# [1] <https://github.com/urllib3/urllib3/issues/651>
release_this_conn = release_conn
http_tunnel_required = connection_requires_http_tunnel(
self.proxy, self.proxy_config, destination_scheme
)
# Merge the proxy headers. Only done when not using HTTP CONNECT. We
# have to copy the headers dict so we can safely change it without those
# changes being reflected in anyone else's copy.
if not http_tunnel_required:
headers = headers.copy()
headers.update(self.proxy_headers)
# Must keep the exception bound to a separate variable or else Python 3
# complains about UnboundLocalError.
err = None
# Keep track of whether we cleanly exited the except block. This
# ensures we do proper cleanup in finally.
clean_exit = False
# Rewind body position, if needed. Record current position
# for future rewinds in the event of a redirect/retry.
body_pos = set_file_position(body, body_pos)
try:
# Request a connection from the queue.
timeout_obj = self._get_timeout(timeout)
conn = self._get_conn(timeout=pool_timeout)
conn.timeout = timeout_obj.connect_timeout
is_new_proxy_conn = self.proxy is not None and not getattr(
conn, "sock", None
)
if is_new_proxy_conn and http_tunnel_required:
self._prepare_proxy(conn)
# Make the request on the httplib connection object.
> httplib_response = self._make_request(
conn,
method,
url,
timeout=timeout_obj,
body=body,
headers=headers,
chunked=chunked,
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:703:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b75ad3280>
conn = <urllib3.connection.HTTPSConnection object at 0x7f8b76022b50>
method = 'POST', url = '/user/credentials/cdis/access_token'
timeout = Timeout(connect=None, read=None, total=None), chunked = False
httplib_request_kw = {'body': b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwd...p, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}}
timeout_obj = Timeout(connect=None, read=None, total=None)
def _make_request(
self, conn, method, url, timeout=_Default, chunked=False, **httplib_request_kw
):
"""
Perform a request on a given urllib connection object taken from our
pool.
:param conn:
a connection from one of our connection pools
:param timeout:
Socket timeout in seconds for the request. This can be a
float or integer, which will set the same timeout value for
the socket connect and the socket read, or an instance of
:class:`urllib3.util.Timeout`, which gives you more fine-grained
control over your timeouts.
"""
self.num_requests += 1
timeout_obj = self._get_timeout(timeout)
timeout_obj.start_connect()
conn.timeout = timeout_obj.connect_timeout
# Trigger any extra validation we need to do.
try:
> self._validate_conn(conn)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:386:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b75ad3280>
conn = <urllib3.connection.HTTPSConnection object at 0x7f8b76022b50>
def _validate_conn(self, conn):
"""
Called right before a request is made, after the socket is created.
"""
super(HTTPSConnectionPool, self)._validate_conn(conn)
# Force connect early to allow us to validate the connection.
if not getattr(conn, "sock", None): # AppEngine might not have `.sock`
> conn.connect()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:1042:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connection.HTTPSConnection object at 0x7f8b76022b50>
def connect(self):
# Add certificate verification
> self.sock = conn = self._new_conn()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:358:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connection.HTTPSConnection object at 0x7f8b76022b50>
def _new_conn(self):
"""Establish a socket connection and set nodelay settings on it.
:return: New socket connection.
"""
extra_kw = {}
if self.source_address:
extra_kw["source_address"] = self.source_address
if self.socket_options:
extra_kw["socket_options"] = self.socket_options
try:
conn = connection.create_connection(
(self._dns_host, self.port), self.timeout, **extra_kw
)
except SocketTimeout:
raise ConnectTimeoutError(
self,
"Connection to %s timed out. (connect timeout=%s)"
% (self.host, self.timeout),
)
except SocketError as e:
> raise NewConnectionError(
self, "Failed to establish a new connection: %s" % e
)
E urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f8b76022b50>: Failed to establish a new connection: [Errno 110] Connection timed out
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connection.py:186: NewConnectionError
During handling of the above exception, another exception occurred:
self = <requests.adapters.HTTPAdapter object at 0x7f8b75ad3610>
request = <PreparedRequest [POST]>, stream = False
timeout = Timeout(connect=None, read=None, total=None), verify = True
cert = None, proxies = OrderedDict()
def send(
self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None
):
"""Sends PreparedRequest object. Returns Response object.
:param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
:param stream: (optional) Whether to stream the request content.
:param timeout: (optional) How long to wait for the server to send
data before giving up, as a float, or a :ref:`(connect timeout,
read timeout) <timeouts>` tuple.
:type timeout: float or tuple or urllib3 Timeout object
:param verify: (optional) Either a boolean, in which case it controls whether
we verify the server's TLS certificate, or a string, in which case it
must be a path to a CA bundle to use
:param cert: (optional) Any user-provided SSL certificate to be trusted.
:param proxies: (optional) The proxies dictionary to apply to the request.
:rtype: requests.Response
"""
try:
conn = self.get_connection(request.url, proxies)
except LocationValueError as e:
raise InvalidURL(e, request=request)
self.cert_verify(conn, request.url, verify, cert)
url = self.request_url(request, proxies)
self.add_headers(
request,
stream=stream,
timeout=timeout,
verify=verify,
cert=cert,
proxies=proxies,
)
chunked = not (request.body is None or "Content-Length" in request.headers)
if isinstance(timeout, tuple):
try:
connect, read = timeout
timeout = TimeoutSauce(connect=connect, read=read)
except ValueError:
raise ValueError(
f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, "
f"or a single float to set both timeouts to the same value."
)
elif isinstance(timeout, TimeoutSauce):
pass
else:
timeout = TimeoutSauce(connect=timeout, read=timeout)
try:
if not chunked:
> resp = conn.urlopen(
method=request.method,
url=url,
body=request.body,
headers=request.headers,
redirect=False,
assert_same_host=False,
preload_content=False,
decode_content=False,
retries=self.max_retries,
timeout=timeout,
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:489:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b75ad3280>
method = 'POST', url = '/user/credentials/cdis/access_token'
body = b'{"api_key": "eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImtpZCI6ImZlbmNlX2tleV8yMDIyLTA4LTIzVDE5OjI4OjM0WiJ9.eyJwdXIiOiJhcG...vPrcq8DT5FB92JLhLStE4nkB9Ysm96h77xFJ44rGq3TODjZaY6Tdj1JgHQrf8BoV2Q", "key_id": "cc7139d4-a9a2-45db-98f8-309087b6b674"}'
headers = {'User-Agent': 'python-requests/2.28.2', 'Accept-Encoding': 'gzip, deflate', 'Accept': '*/*', 'Connection': 'keep-alive', 'Content-Length': '771', 'Content-Type': 'application/json'}
retries = Retry(total=0, connect=None, read=False, redirect=None, status=None)
redirect = False, assert_same_host = False
timeout = Timeout(connect=None, read=None, total=None), pool_timeout = None
release_conn = False, chunked = False, body_pos = None
response_kw = {'decode_content': False, 'preload_content': False}
parsed_url = Url(scheme=None, auth=None, host=None, port=None, path='/user/credentials/cdis/access_token', query=None, fragment=None)
destination_scheme = None, conn = None, release_this_conn = True
http_tunnel_required = False, err = None, clean_exit = False
def urlopen(
self,
method,
url,
body=None,
headers=None,
retries=None,
redirect=True,
assert_same_host=True,
timeout=_Default,
pool_timeout=None,
release_conn=None,
chunked=False,
body_pos=None,
**response_kw
):
"""
Get a connection from the pool and perform an HTTP request. This is the
lowest level call for making a request, so you'll need to specify all
the raw details.
.. note::
More commonly, it's appropriate to use a convenience method provided
by :class:`.RequestMethods`, such as :meth:`request`.
.. note::
`release_conn` will only behave as expected if
`preload_content=False` because we want to make
`preload_content=False` the default behaviour someday soon without
breaking backwards compatibility.
:param method:
HTTP request method (such as GET, POST, PUT, etc.)
:param url:
The URL to perform the request on.
:param body:
Data to send in the request body, either :class:`str`, :class:`bytes`,
an iterable of :class:`str`/:class:`bytes`, or a file-like object.
:param headers:
Dictionary of custom headers to send, such as User-Agent,
If-None-Match, etc. If None, pool headers are used. If provided,
these headers completely replace any pool-specific headers.
:param retries:
Configure the number of retries to allow before raising a
:class:`~urllib3.exceptions.MaxRetryError` exception.
Pass ``None`` to retry until you receive a response. Pass a
:class:`~urllib3.util.retry.Retry` object for fine-grained control
over different types of retries.
Pass an integer number to retry connection errors that many times,
but no other types of errors. Pass zero to never retry.
If ``False``, then retries are disabled and any exception is raised
immediately. Also, instead of raising a MaxRetryError on redirects,
the redirect response will be returned.
:type retries: :class:`~urllib3.util.retry.Retry`, False, or an int.
:param redirect:
If True, automatically handle redirects (status codes 301, 302,
303, 307, 308). Each redirect counts as a retry. Disabling retries
will disable redirect, too.
:param assert_same_host:
If ``True``, will make sure that the host of the pool requests is
consistent else will raise HostChangedError. When ``False``, you can
use the pool on an HTTP proxy and request foreign hosts.
:param timeout:
If specified, overrides the default timeout for this one
request. It may be a float (in seconds) or an instance of
:class:`urllib3.util.Timeout`.
:param pool_timeout:
If set and the pool is set to block=True, then this method will
block for ``pool_timeout`` seconds and raise EmptyPoolError if no
connection is available within the time period.
:param release_conn:
If False, then the urlopen call will not release the connection
back into the pool once a response is received (but will release if
you read the entire contents of the response such as when
`preload_content=True`). This is useful if you're not preloading
the response's content immediately. You will need to call
``r.release_conn()`` on the response ``r`` to return the connection
back into the pool. If None, it takes the value of
``response_kw.get('preload_content', True)``.
:param chunked:
If True, urllib3 will send the body using chunked transfer
encoding. Otherwise, urllib3 will send the body using the standard
content-length form. Defaults to False.
:param int body_pos:
Position to seek to in file-like body in the event of a retry or
redirect. Typically this won't need to be set because urllib3 will
auto-populate the value when needed.
:param \\**response_kw:
Additional parameters are passed to
:meth:`urllib3.response.HTTPResponse.from_httplib`
"""
parsed_url = parse_url(url)
destination_scheme = parsed_url.scheme
if headers is None:
headers = self.headers
if not isinstance(retries, Retry):
retries = Retry.from_int(retries, redirect=redirect, default=self.retries)
if release_conn is None:
release_conn = response_kw.get("preload_content", True)
# Check host
if assert_same_host and not self.is_same_host(url):
raise HostChangedError(self, url, retries)
# Ensure that the URL we're connecting to is properly encoded
if url.startswith("/"):
url = six.ensure_str(_encode_target(url))
else:
url = six.ensure_str(parsed_url.url)
conn = None
# Track whether `conn` needs to be released before
# returning/raising/recursing. Update this variable if necessary, and
# leave `release_conn` constant throughout the function. That way, if
# the function recurses, the original value of `release_conn` will be
# passed down into the recursive call, and its value will be respected.
#
# See issue #651 [1] for details.
#
# [1] <https://github.com/urllib3/urllib3/issues/651>
release_this_conn = release_conn
http_tunnel_required = connection_requires_http_tunnel(
self.proxy, self.proxy_config, destination_scheme
)
# Merge the proxy headers. Only done when not using HTTP CONNECT. We
# have to copy the headers dict so we can safely change it without those
# changes being reflected in anyone else's copy.
if not http_tunnel_required:
headers = headers.copy()
headers.update(self.proxy_headers)
# Must keep the exception bound to a separate variable or else Python 3
# complains about UnboundLocalError.
err = None
# Keep track of whether we cleanly exited the except block. This
# ensures we do proper cleanup in finally.
clean_exit = False
# Rewind body position, if needed. Record current position
# for future rewinds in the event of a redirect/retry.
body_pos = set_file_position(body, body_pos)
try:
# Request a connection from the queue.
timeout_obj = self._get_timeout(timeout)
conn = self._get_conn(timeout=pool_timeout)
conn.timeout = timeout_obj.connect_timeout
is_new_proxy_conn = self.proxy is not None and not getattr(
conn, "sock", None
)
if is_new_proxy_conn and http_tunnel_required:
self._prepare_proxy(conn)
# Make the request on the httplib connection object.
httplib_response = self._make_request(
conn,
method,
url,
timeout=timeout_obj,
body=body,
headers=headers,
chunked=chunked,
)
# If we're going to release the connection in ``finally:``, then
# the response doesn't need to know about the connection. Otherwise
# it will also try to release it and we'll have a double-release
# mess.
response_conn = conn if not release_conn else None
# Pass method to Response for length checking
response_kw["request_method"] = method
# Import httplib's response into our own wrapper object
response = self.ResponseCls.from_httplib(
httplib_response,
pool=self,
connection=response_conn,
retries=retries,
**response_kw
)
# Everything went great!
clean_exit = True
except EmptyPoolError:
# Didn't get a connection from the pool, no need to clean up
clean_exit = True
release_this_conn = False
raise
except (
TimeoutError,
HTTPException,
SocketError,
ProtocolError,
BaseSSLError,
SSLError,
CertificateError,
) as e:
# Discard the connection for these exceptions. It will be
# replaced during the next _get_conn() call.
clean_exit = False
def _is_ssl_error_message_from_http_proxy(ssl_error):
# We're trying to detect the message 'WRONG_VERSION_NUMBER' but
# SSLErrors are kinda all over the place when it comes to the message,
# so we try to cover our bases here!
message = " ".join(re.split("[^a-z]", str(ssl_error).lower()))
return (
"wrong version number" in message or "unknown protocol" in message
)
# Try to detect a common user error with proxies which is to
# set an HTTP proxy to be HTTPS when it should be 'http://'
# (ie {'http': 'http://proxy', 'https': 'https://proxy'})
# Instead we add a nice error message and point to a URL.
if (
isinstance(e, BaseSSLError)
and self.proxy
and _is_ssl_error_message_from_http_proxy(e)
and conn.proxy
and conn.proxy.scheme == "https"
):
e = ProxyError(
"Your proxy appears to only use HTTP and not HTTPS, "
"try changing your proxy URL to be HTTP. See: "
"https://urllib3.readthedocs.io/en/1.26.x/advanced-usage.html"
"#https-proxy-error-http-proxy",
SSLError(e),
)
elif isinstance(e, (BaseSSLError, CertificateError)):
e = SSLError(e)
elif isinstance(e, (SocketError, NewConnectionError)) and self.proxy:
e = ProxyError("Cannot connect to proxy.", e)
elif isinstance(e, (SocketError, HTTPException)):
e = ProtocolError("Connection aborted.", e)
> retries = retries.increment(
method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/connectionpool.py:787:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = Retry(total=0, connect=None, read=False, redirect=None, status=None)
method = 'POST', url = '/user/credentials/cdis/access_token', response = None
error = NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8b76022b50>: Failed to establish a new connection: [Errno 110] Connection timed out')
_pool = <urllib3.connectionpool.HTTPSConnectionPool object at 0x7f8b75ad3280>
_stacktrace = <traceback object at 0x7f8b75f30e00>
def increment(
self,
method=None,
url=None,
response=None,
error=None,
_pool=None,
_stacktrace=None,
):
"""Return a new Retry object with incremented retry counters.
:param response: A response object, or None, if the server did not
return a response.
:type response: :class:`~urllib3.response.HTTPResponse`
:param Exception error: An error encountered during the request, or
None if the response was received successfully.
:return: A new ``Retry`` object.
"""
if self.total is False and error:
# Disabled, indicate to re-raise the error.
raise six.reraise(type(error), error, _stacktrace)
total = self.total
if total is not None:
total -= 1
connect = self.connect
read = self.read
redirect = self.redirect
status_count = self.status
other = self.other
cause = "unknown"
status = None
redirect_location = None
if error and self._is_connection_error(error):
# Connect retry?
if connect is False:
raise six.reraise(type(error), error, _stacktrace)
elif connect is not None:
connect -= 1
elif error and self._is_read_error(error):
# Read retry?
if read is False or not self._is_method_retryable(method):
raise six.reraise(type(error), error, _stacktrace)
elif read is not None:
read -= 1
elif error:
# Other retry?
if other is not None:
other -= 1
elif response and response.get_redirect_location():
# Redirect retry?
if redirect is not None:
redirect -= 1
cause = "too many redirects"
redirect_location = response.get_redirect_location()
status = response.status
else:
# Incrementing because of a server error like a 500 in
# status_forcelist and the given method is in the allowed_methods
cause = ResponseError.GENERIC_ERROR
if response and response.status:
if status_count is not None:
status_count -= 1
cause = ResponseError.SPECIFIC_ERROR.format(status_code=response.status)
status = response.status
history = self.history + (
RequestHistory(method, url, error, status, redirect_location),
)
new_retry = self.new(
total=total,
connect=connect,
read=read,
redirect=redirect,
status=status_count,
other=other,
history=history,
)
if new_retry.is_exhausted():
> raise MaxRetryError(_pool, url, error or ResponseError(cause))
E urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8b76022b50>: Failed to establish a new connection: [Errno 110] Connection timed out'))
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/urllib3/util/retry.py:592: MaxRetryError
During handling of the above exception, another exception occurred:
client = <starlette.testclient.TestClient object at 0x7f8b76aafa90>
def test_case_node(client):
UUID = "c58ab983-6cf9-4174-a7a9-20cdf1d6bc33"
payload = {
"program": "demo1",
"project": "12L",
}
> response = client.post(
f"/record/{UUID}", json=payload)
tests/test_data.py:219:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:635: in post
return self.request("POST", url, data=data, json=json, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:476: in request
return super().request(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:587: in request
resp = self.send(prep, **send_kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:701: in send
r = adapter.send(request, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:270: in send
raise exc
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/testclient.py:267: in send
portal.call(self.app, scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:283: in call
return cast(T_Retval, self.start_task_soon(func, *args).result())
/usr/lib/python3.9/concurrent/futures/_base.py:446: in result
return self.__get_result()
/usr/lib/python3.9/concurrent/futures/_base.py:391: in __get_result
raise self._exception
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/anyio/from_thread.py:219: in _call_func
retval = await retval
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/applications.py:269: in __call__
await super().__call__(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/applications.py:124: in __call__
await self.middleware_stack(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:184: in __call__
raise exc
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/errors.py:162: in __call__
await self.app(scope, receive, _send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/middleware/cors.py:84: in __call__
await self.app(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:93: in __call__
raise exc
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/exceptions.py:82: in __call__
await self.app(scope, receive, sender)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:21: in __call__
raise e
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/middleware/asyncexitstack.py:18: in __call__
await self.app(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:670: in __call__
await route.handle(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:266: in handle
await self.app(scope, receive, send)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/starlette/routing.py:65: in app
response = await func(request)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:227: in app
raw_response = await run_endpoint_function(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/fastapi/routing.py:160: in run_endpoint_function
return await dependant.call(**values)
app/main.py:218: in get_gen3_record
record = SUBMISSION.export_record(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/submission.py:348: in export_record
output = requests.get(api_url, auth=self._auth_provider).text
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:73: in get
return request("get", url, params=params, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request
return session.request(method=method, url=url, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:573: in request
prep = self.prepare_request(req)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:484: in prepare_request
p.prepare(
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:372: in prepare
self.prepare_auth(auth, url)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/models.py:603: in prepare_auth
r = auth(self)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:266: in __call__
request.headers["Authorization"] = self._get_auth_value()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:378: in _get_auth_value
return "bearer " + self.get_access_token()
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:367: in get_access_token
return self.refresh_access_token(endpoint)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:304: in refresh_access_token
self._access_token = get_access_token_with_key(self._refresh_token)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/gen3/auth.py:75: in get_access_token_with_key
resp = requests.post(auth_url, json=api_key)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:115: in post
return request("post", url, data=data, json=json, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/api.py:59: in request
return session.request(method=method, url=url, **kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:587: in request
resp = self.send(prep, **send_kwargs)
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/sessions.py:701: in send
r = adapter.send(request, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <requests.adapters.HTTPAdapter object at 0x7f8b75ad3610>
request = <PreparedRequest [POST]>, stream = False
timeout = Timeout(connect=None, read=None, total=None), verify = True
cert = None, proxies = OrderedDict()
def send(
self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None
):
"""Sends PreparedRequest object. Returns Response object.
:param request: The :class:`PreparedRequest <PreparedRequest>` being sent.
:param stream: (optional) Whether to stream the request content.
:param timeout: (optional) How long to wait for the server to send
data before giving up, as a float, or a :ref:`(connect timeout,
read timeout) <timeouts>` tuple.
:type timeout: float or tuple or urllib3 Timeout object
:param verify: (optional) Either a boolean, in which case it controls whether
we verify the server's TLS certificate, or a string, in which case it
must be a path to a CA bundle to use
:param cert: (optional) Any user-provided SSL certificate to be trusted.
:param proxies: (optional) The proxies dictionary to apply to the request.
:rtype: requests.Response
"""
try:
conn = self.get_connection(request.url, proxies)
except LocationValueError as e:
raise InvalidURL(e, request=request)
self.cert_verify(conn, request.url, verify, cert)
url = self.request_url(request, proxies)
self.add_headers(
request,
stream=stream,
timeout=timeout,
verify=verify,
cert=cert,
proxies=proxies,
)
chunked = not (request.body is None or "Content-Length" in request.headers)
if isinstance(timeout, tuple):
try:
connect, read = timeout
timeout = TimeoutSauce(connect=connect, read=read)
except ValueError:
raise ValueError(
f"Invalid timeout {timeout}. Pass a (connect, read) timeout tuple, "
f"or a single float to set both timeouts to the same value."
)
elif isinstance(timeout, TimeoutSauce):
pass
else:
timeout = TimeoutSauce(connect=timeout, read=timeout)
try:
if not chunked:
resp = conn.urlopen(
method=request.method,
url=url,
body=request.body,
headers=request.headers,
redirect=False,
assert_same_host=False,
preload_content=False,
decode_content=False,
retries=self.max_retries,
timeout=timeout,
)
# Send the request.
else:
if hasattr(conn, "proxy_pool"):
conn = conn.proxy_pool
low_conn = conn._get_conn(timeout=DEFAULT_POOL_TIMEOUT)
try:
skip_host = "Host" in request.headers
low_conn.putrequest(
request.method,
url,
skip_accept_encoding=True,
skip_host=skip_host,
)
for header, value in request.headers.items():
low_conn.putheader(header, value)
low_conn.endheaders()
for i in request.body:
low_conn.send(hex(len(i))[2:].encode("utf-8"))
low_conn.send(b"\r\n")
low_conn.send(i)
low_conn.send(b"\r\n")
low_conn.send(b"0\r\n\r\n")
# Receive the response from the server
r = low_conn.getresponse()
resp = HTTPResponse.from_httplib(
r,
pool=conn,
connection=low_conn,
preload_content=False,
decode_content=False,
)
except Exception:
# If we hit any problems here, clean up the connection.
# Then, raise so that we can handle the actual exception.
low_conn.close()
raise
except (ProtocolError, OSError) as err:
raise ConnectionError(err, request=request)
except MaxRetryError as e:
if isinstance(e.reason, ConnectTimeoutError):
# TODO: Remove this in 3.0.0: see #2811
if not isinstance(e.reason, NewConnectionError):
raise ConnectTimeout(e, request=request)
if isinstance(e.reason, ResponseError):
raise RetryError(e, request=request)
if isinstance(e.reason, _ProxyError):
raise ProxyError(e, request=request)
if isinstance(e.reason, _SSLError):
# This branch is for urllib3 v1.22 and later.
raise SSLError(e, request=request)
> raise ConnectionError(e, request=request)
E requests.exceptions.ConnectionError: HTTPSConnectionPool(host='gen3.abi-ctt-ctp.cloud.edu.au', port=443): Max retries exceeded with url: /user/credentials/cdis/access_token (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f8b76022b50>: Failed to establish a new connection: [Errno 110] Connection timed out'))
../../shiningpanda/jobs/084398e7/virtualenvs/d41d8cd9/lib/python3.9/site-packages/requests/adapters.py:565: ConnectionError
=========================== short test summary info ============================
FAILED tests/test_api.py::test_get_gen3_program - assert 404 == 200
FAILED tests/test_api.py::test_get_gen3_project - assert 404 == 200
FAILED tests/test_api.py::test_get_gen3_dictionary - assert 404 == 200
FAILED tests/test_api.py::test_get_gen3_node_records - requests.exceptions.Co...
FAILED tests/test_api.py::test_get_gen3_record - requests.exceptions.Connecti...
FAILED tests/test_api.py::test_graphql_query - assert 404 == 200
FAILED tests/test_api.py::test_graphql_pagination - assert 404 == 200
FAILED tests/test_api.py::test_download_gen3_metadata_file - assert 400 == 200
FAILED tests/test_data.py::test_experiment_node - requests.exceptions.Connect...
FAILED tests/test_data.py::test_dataset_description_node - requests.exception...
FAILED tests/test_data.py::test_manifest_node - requests.exceptions.Connectio...
FAILED tests/test_data.py::test_case_node - requests.exceptions.ConnectionErr...
================== 12 failed, 5 passed in 1573.57s (0:26:13) ===================
Build step 'Virtualenv Builder' marked build as failure
[Slack Notifications] found #464 as previous completed, non-aborted build
[Slack Notifications] will send OnEveryFailureNotification because build matches and user preferences allow it
Finished: FAILURE