Uploaded image for project: 'SimplyE 2.0'
  1. SimplyE 2.0
  2. SIMPLY-2513

Improve the efficiency of reapers

XMLWordPrintable

    • SIMPLY S3 January 21 - Feb 4

      We've seen that reapers may take a long time to run and sometimes even fail because they try to load an entire queryset into memory. But most reapers don't do anything with that queryset except iterate over it and delete every row. This could be done much more efficiently with a single DELETE statement – it's just a little awkward to write in SQLAlchemy.

      ScrubberMonitor does something similar with a single UPDATE statement, so the awkward part has been mostly figured out. We could copy this code and use it for most, but probably not all, ReaperMonitors.

            leonardrichardson Leonard Richardson [X] (Inactive)
            leonardrichardson Leonard Richardson [X] (Inactive)
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

              Created:
              Updated:
              Resolved: