3 P\@sdZddlZddlZddlmZddlmZGdddeZGd d d eZ Gd d d e Z Gd dde Z ddZ dd dgZ dS)zEModule implementing the Pool for :mod:``requests_toolbelt.threaded``.N)thread)queuec@sneZdZdZdddejfddZddZeddZ edd d Z d d Z d dZ ddZ ddZddZdS)Poola>Pool that manages the threads containing sessions. :param queue: The queue you're expected to use to which you should add items. :type queue: queue.Queue :param initializer: Function used to initialize an instance of ``session``. :type initializer: collections.Callable :param auth_generator: Function used to generate new auth credentials for the session. :type auth_generator: collections.Callable :param int num_process: Number of threads to create. :param session: :type session: requests.Session Ncs||dkrtjpd}|dkr$td|_tj_tj_|_|pJt _ |pTt _ |_ fddt jD_dS)Nrz)Number of processes should at least be 1.cs&g|]}tjjjjjqS)rZ SessionThread _new_session _job_queue_response_queue _exc_queue).0_)selfr/usr/lib/python3.6/pool.py +sz!Pool.__init__..)multiprocessing cpu_count ValueErrorr rQueuer r Z _processes _identity _initializer_auth_sessionrange_pool)r job_queueZ initializerZauth_generatorZ num_processesZsessionr)rr__init__s      z Pool.__init__cCs|j|j|jS)N)rrr)rrrrr0szPool._new_sessioncKs4tj}x|D]}|j|jqW|fd|i|S)a2Create a :class:`~Pool` from an :class:`~ThreadException`\ s. Provided an iterable that provides :class:`~ThreadException` objects, this classmethod will generate a new pool to retry the requests that caused the exceptions. :param exceptions: Iterable that returns :class:`~ThreadException` :type exceptions: iterable :param kwargs: Keyword arguments passed to the :class:`~Pool` initializer. :returns: An initialized :class:`~Pool` object. :rtype: :class:`~Pool` r)rrputrequest_kwargs)cls exceptionskwargsrexcrrrfrom_exceptions3s zPool.from_exceptionscKs^ddi}|j|pitj}x,|D]$}|j}|jd|i|j|q$W|fd|i|S)aCreate a :class:`~Pool` from an iterable of URLs. :param urls: Iterable that returns URLs with which we create a pool. :type urls: iterable :param dict request_kwargs: Dictionary of other keyword arguments to provide to the request method. :param kwargs: Keyword arguments passed to the :class:`~Pool` initializer. :returns: An initialized :class:`~Pool` object. :rtype: :class:`~Pool` methodZGETurlr)updaterrcopyr)rZurlsrr!Z request_dictrr%Zjobrrr from_urlsIs zPool.from_urlsccs"x|j}|dkrP|VqWdS)zoIterate over all the exceptions in the pool. :returns: Generator of :class:`~ThreadException` N) get_exception)rr"rrrr bs zPool.exceptionsc Cs8y|jj\}}Wntjk r(dSXt||SdS)zSGet an exception from the pool. :rtype: :class:`~ThreadException` N)r get_nowaitrEmptyThreadException)rrequestr"rrrr)ms zPool.get_exceptionc Cs8y|jj\}}Wntjk r(dSXt||SdS)zPGet a response from the pool. :rtype: :class:`~ThreadResponse` N)r r*rr+ThreadResponse)rr-responserrr get_responseys zPool.get_responseccs"x|j}|dkrP|VqWdS)zmIterate over all the responses in the pool. :returns: Generator of :class:`~ThreadResponse` N)r0)rZresprrr responsess zPool.responsescCsx|jD] }|jqWdS)z*Join all the threads to the master thread.N)rjoin)rZsession_threadrrrjoin_alls z Pool.join_all)N)__name__ __module__ __qualname____doc__requestsZSessionrr classmethodr#r(r r)r0r1r3rrrrr s      rc@seZdZdZddZdS) ThreadProxyNcCs4tj}||jkr&|||j}t||S|||SdS)z/Proxy attribute accesses to the proxied object.N)object__getattribute__attrs proxied_attrgetattr)rattrgetr/rrr __getattr__s    zThreadProxy.__getattr__)r4r5r6r>rBrrrrr:sr:c@s(eZdZdZdZeddgZddZdS)r.a>A wrapper around a requests Response object. This will proxy most attribute access actions to the Response object. For example, if you wanted the parsed JSON from the response, you might do: .. code-block:: python thread_response = pool.get_response() json = thread_response.json() r/rcCs||_||_dS)N)rr/)rrr/rrrrszThreadResponse.__init__N)r4r5r6r7r> frozensetr=rrrrrr.s  r.c@s(eZdZdZdZeddgZddZdS)r,a=A wrapper around an exception raised during a request. This will proxy most attribute access actions to the exception object. For example, if you wanted the message from the exception, you might do: .. code-block:: python thread_exc = pool.get_exception() msg = thread_exc.message exceptionrcCs||_||_dS)N)rrD)rrrDrrrrszThreadException.__init__N)r4r5r6r7r>rCr=rrrrrr,s  r,cCs|S)Nr)Z session_objrrrrsr)r7rr8rZ_compatrr;rr:r.r,r__all__rrrrs