Uploaded image for project: 'SimplyE 2.0'
  1. SimplyE 2.0
  2. SIMPLY-3348

Increase timeout for S3 interactions and consider making timeout a site-wide setting

XMLWordPrintable

    • SIMPLY Sprint 26 Dec 8 - 22, SIMPLY S0 Dec 22 - Jan 5, SIMPLY S1 January 5 - 19, SIMPLY S2 Jan 19 - Feb 2, SIMPLY S3 Feb 2 - 16, SIMPLY S4 Feb 16 - March 2, SIMPLY S5 March 2 - March 16, SIMPLY S6 March 16 - March 30, SIMPLY S7 March 30 - April 13, SIMPLY S8 April 13 - 27, SIMPLY S9 April 27 - May 11, SIMPLY S10 May 11 - May 25

      There are a number of different HTTP timeouts present in the circulation manager:

      • core/util/http.py:Http._request_with_timeout allows a timeout to be passed in. The default timeout is 20 seconds.
      • Various API clients that end up calling _request_with_timeout() pass in different timeouts for different servers–we determined these numbers empirically over time.
      • When interacting with S3 through boto, the rules for setting timeouts are different, since boto is an external dependency that doesn't use our Http class. Here's some documentation which says the default timeout is 60 seconds, but in our application it seems to be 20 seconds, the same as the default timeout set by the Http class.

      The immediate problem we're facing is that 20 seconds is not a long enough timeout for boto. It should be at least 60 seconds.

      Beyond that, it may be desirable to create a sitewide setting that sets a global timeout so that 1) we don't have to figure out the best timeout for every service we interact with, 2) an administrator has an easy way to bump up the timeout temporarily if it's causing problems with a specific task.

            leonardrichardson Leonard Richardson [X] (Inactive)
            leonardrichardson Leonard Richardson [X] (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:
              Resolved: