Slow products page on 2.0

I just ran the explain again while the products page was loading… now the products page loads within a reasonable second or two :frowning:

No idea. If this happens again i’ll grab the explain to find out why. Maybe an automated task ran and “fixed” whatever was blocking the queries?

Could it be that the table had a lot of records that were deleted in the past and your explain query forced a recalculation of the statistics or vacuum run?

Hello

i am having the exact same problem. Products page after the upgrade is unbearable ~40s to load it.

2020-06-11T09:36:36 [I|app|ab7d551e] Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.15.1.1/app/views/katello/api/v2/products/index.json.rabl within katello/api/v2/layouts/collection (41740.3ms)
2020-06-11T09:36:36 [I|app|ab7d551e] Completed 200 OK in 41822ms (Views: 713.8ms | ActiveRecord: 41046.1ms)

I also run the query you provided with “EXPLAIN ANALYZE” and it run within:

Planning Time: 2.157 ms
Execution Time: 7143.341 ms

I saw that you suggested to remove old tasks which would maybe help with the performance. I should have total of ~63k tasks but when i try to perform the cleanup i am getting an error from:

[root@ foreman]# foreman-rake foreman_tasks:cleanup TASK_SEARCH=‘label ~ *’ AFTER=‘1d’ VERBOSE=true
rake aborted!
The Dynflow world was not initialized yet. If your plugin uses it, make sure to call Rails.application.dynflow.require! in some initializer
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.3/lib/dynflow/rails.rb:75:in world' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-1.1.1/lib/foreman_tasks/cleaner.rb:80:in initialize’
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-1.1.1/lib/foreman_tasks/cleaner.rb:8:in new' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-1.1.1/lib/foreman_tasks/cleaner.rb:8:in run’
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-1.1.1/lib/foreman_tasks/tasks/cleanup.rake:37:in block (3 levels) in <top (required)>' /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in <top (required)>’
Tasks: TOP => foreman_tasks:cleanup => foreman_tasks:cleanup:run
(See full trace by running task with --trace)

Note that i upgraded from:
Foreman 1.24.3 -> 2.0.1
Katello 3.14.1 -> 3.15.1.1

Turned also on debugging and top SQL consumers by time are these:

2020-06-11T10:08:04 [D|sql|4b64480e] ForemanTasks::Task::DynflowTask Load (6890.4ms) SELECT DISTINCT foreman_tasks_tasks., inner_select.resource_id FROM “foreman_tasks_tasks” INNER JOIN “foreman_tasks_locks” “locks_foreman_tasks_tasks” ON “locks_foreman_tasks_tasks”.“task_id” = “foreman_tasks_tasks”.“id” INNER JOIN (SELECT MAX(foreman_tasks_tasks.started_at) AS started_at, foreman_tasks_locks.resource_id AS resource_id FROM “foreman_tasks_tasks” INNER JOIN “foreman_tasks_locks” ON “foreman_tasks_locks”.“task_id” = “foreman_tasks_tasks”.“id” WHERE “foreman_tasks_tasks”.“type” IN (‘ForemanTasks::Task::DynflowTask’) AND “foreman_tasks_tasks”.“label” = ‘Actions::Katello::Repository::Sync’ AND (foreman_tasks_locks.resource_id in (59,516,517,518,520,519,8203,8241) and foreman_tasks_locks.resource_type = ‘Katello::Repository’) GROUP BY foreman_tasks_locks.resource_id) inner_select
2020-06-11T10:08:11 [D|sql|4b64480e] ForemanTasks::Task::DynflowTask Load (7093.2ms) SELECT foreman_tasks_tasks.
FROM “foreman_tasks_tasks” INNER JOIN “foreman_tasks_locks” ON “foreman_tasks_locks”.“task_id” = “foreman_tasks_tasks”.“id” WHERE “foreman_tasks_tasks”.“type” IN (‘ForemanTasks::Task::DynflowTask’) AND “foreman_tasks_tasks”.“label” = $1 AND (foreman_tasks_locks.resource_id in (59,516,517,518,520,519,8203,8241) and foreman_tasks_locks.resource_type = ‘Katello::Repository’) ORDER BY started_at desc LIMIT $2 [[“label”, “Actions::Katello::Repository::Sync”], [“LIMIT”, 1]]
2020-06-11T10:08:18 [D|sql|4b64480e] ForemanTasks::Task::DynflowTask Load (6703.9ms) SELECT DISTINCT foreman_tasks_tasks., inner_select.resource_id FROM “foreman_tasks_tasks” INNER JOIN “foreman_tasks_locks” “locks_foreman_tasks_tasks” ON “locks_foreman_tasks_tasks”.“task_id” = “foreman_tasks_tasks”.“id” INNER JOIN (SELECT MAX(foreman_tasks_tasks.started_at) AS started_at, foreman_tasks_locks.resource_id AS resource_id FROM “foreman_tasks_tasks” INNER JOIN “foreman_tasks_locks” ON “foreman_tasks_locks”.“task_id” = “foreman_tasks_tasks”.“id” WHERE “foreman_tasks_tasks”.“type” IN (‘ForemanTasks::Task::DynflowTask’) AND “foreman_tasks_tasks”.“label” = ‘Actions::Katello::Repository::Sync’ AND (foreman_tasks_locks.resource_id in (57,10,9,60,1361,8,7,6590,8202,8240) and foreman_tasks_locks.resource_type = ‘Katello::Repository’) GROUP BY foreman_tasks_locks.resource_id) inner_select
2020-06-11T10:08:24 [D|sql|4b64480e] ForemanTasks::Task::DynflowTask Load (6666.9ms) SELECT foreman_tasks_tasks.
FROM “foreman_tasks_tasks” INNER JOIN “foreman_tasks_locks” ON “foreman_tasks_locks”.“task_id” = “foreman_tasks_tasks”.“id” WHERE “foreman_tasks_tasks”.“type” IN (‘ForemanTasks::Task::DynflowTask’) AND “foreman_tasks_tasks”.“label” = $1 AND (foreman_tasks_locks.resource_id in (57,10,9,60,1361,8,7,6590,8202,8240) and foreman_tasks_locks.resource_type = ‘Katello::Repository’) ORDER BY started_at desc LIMIT $2 [[“label”, “Actions::Katello::Repository::Sync”], [“LIMIT”, 1]]
2020-06-11T10:08:31 [D|sql|4b64480e] ForemanTasks::Task::DynflowTask Load (6877.8ms) SELECT DISTINCT foreman_tasks_tasks., inner_select.resource_id FROM “foreman_tasks_tasks” INNER JOIN “foreman_tasks_locks” “locks_foreman_tasks_tasks” ON “locks_foreman_tasks_tasks”.“task_id” = “foreman_tasks_tasks”.“id” INNER JOIN (SELECT MAX(foreman_tasks_tasks.started_at) AS started_at, foreman_tasks_locks.resource_id AS resource_id FROM “foreman_tasks_tasks” INNER JOIN “foreman_tasks_locks” ON “foreman_tasks_locks”.“task_id” = “foreman_tasks_tasks”.“id” WHERE “foreman_tasks_tasks”.“type” IN (‘ForemanTasks::Task::DynflowTask’) AND “foreman_tasks_tasks”.“label” = ‘Actions::Katello::Repository::Sync’ AND (foreman_tasks_locks.resource_id in (10309,10310,10311,10312,10313,10314,10315,10316,10349) and foreman_tasks_locks.resource_type = ‘Katello::Repository’) GROUP BY foreman_tasks_locks.resource_id) inner_select
2020-06-11T10:08:38 [D|sql|4b64480e] ForemanTasks::Task::DynflowTask Load (6558.7ms) SELECT foreman_tasks_tasks.
FROM “foreman_tasks_tasks” INNER JOIN “foreman_tasks_locks” ON “foreman_tasks_locks”.“task_id” = “foreman_tasks_tasks”.“id” WHERE “foreman_tasks_tasks”.“type” IN (‘ForemanTasks::Task::DynflowTask’) AND “foreman_tasks_tasks”.“label” = $1 AND (foreman_tasks_locks.resource_id in (10309,10310,10311,10312,10313,10314,10315,10316,10349) and foreman_tasks_locks.resource_type = ‘Katello::Repository’) ORDER BY started_at desc LIMIT $2 [[“label”, “Actions::Katello::Repository::Sync”], [“LIMIT”, 1]]

End result:

2020-06-11T10:08:38 [I|app|4b64480e] Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.15.1.1/app/views/katello/api/v2/products/index.json.rabl within katello/api/v2/layouts/collection (41003.5ms)

Can you please share the full output of the EXPLAIN ANALYZE for one of those slow queries? Also, can you check if running VACUUM FULL on your db improves the speed?

Running full vacuum was one of the first thing i did and it did not help anything.

su - postgres -c ‘vacuumdb --full --dbname=foreman’

Full output of EXPLAIN ANALYZE:

foreman=> EXPLAIN ANALYZE SELECT DISTINCT foreman_tasks_tasks.*, inner_select.resource_id FROM “foreman_tasks_tasks” INNER JOIN “foreman_tasks_locks” “locks_foreman_tasks_tasks” ON “locks_foreman_tasks_tasks”.“task_id” = “foreman_tasks_tasks”.“id” INNER JOIN (SELECT MAX(foreman_tasks_tasks.started_at) AS started_at, foreman_tasks_locks.resource_id AS resource_id FROM “foreman_tasks_tasks” INNER JOIN “foreman_tasks_locks” ON “foreman_tasks_locks”.“task_id” = “foreman_tasks_tasks”.“id” WHERE “foreman_tasks_tasks”.“type” IN (‘ForemanTasks::Task::DynflowTask’) AND “foreman_tasks_tasks”.“label” = ‘Actions::Katello::Repository::Sync’ AND (foreman_tasks_locks.resource_id in (59,516,517,518,520,519,8203,8241) and foreman_tasks_locks.resource_type = ‘Katello::Repository’) GROUP BY foreman_tasks_locks.resource_id) inner_select
foreman-> ON inner_select.started_at = foreman_tasks_tasks.started_at
foreman-> AND inner_select.resource_id = locks_foreman_tasks_tasks.resource_id WHERE “foreman_tasks_tasks”.“type” IN (‘ForemanTasks::Task::DynflowTask’);
QUERY PLAN


----------------------------------------------------------------------------------------------------------------------------------
 Unique  (cost=203.36..203.51 rows=4 width=272) (actual time=6960.560..6960.573 rows=8 loops=1)
   ->  Sort  (cost=203.36..203.37 rows=4 width=272) (actual time=6960.559..6960.560 rows=16 loops=1)
         Sort Key: foreman_tasks_tasks.id, foreman_tasks_tasks.label, foreman_tasks_tasks.started_at, foreman_tasks_tasks.ended_at, foreman_tasks_tasks.state, foreman_tasks_tasks.result, foreman_tasks_tasks.external_id, foreman_tasks_tasks.parent_task_id, foreman_tasks_tasks.start_at, foreman_tasks_tasks.start_befor
e, foreman_tasks_tasks.action, foreman_tasks_tasks.state_updated_at, foreman_tasks_tasks.user_id, foreman_tasks_locks.resource_id
         Sort Method: quicksort  Memory: 33kB
         ->  Nested Loop  (cost=120.65..203.32 rows=4 width=272) (actual time=6934.744..6960.504 rows=16 loops=1)
               Join Filter: (foreman_tasks_locks.resource_id = locks_foreman_tasks_tasks.resource_id)
               Rows Removed by Join Filter: 24
               ->  Nested Loop  (cost=120.23..128.12 rows=2 width=272) (actual time=6934.731..6960.419 rows=8 loops=1)
                     ->  GroupAggregate  (cost=102.47..102.49 rows=1 width=12) (actual time=6930.797..6931.327 rows=8 loops=1)
                           Group Key: foreman_tasks_locks.resource_id
                           ->  Sort  (cost=102.47..102.48 rows=1 width=12) (actual time=6930.708..6930.975 rows=3170 loops=1)
                                 Sort Key: foreman_tasks_locks.resource_id
                                 Sort Method: quicksort  Memory: 245kB
                                 ->  Nested Loop  (cost=37.90..102.46 rows=1 width=12) (actual time=14.587..6929.493 rows=3170 loops=1)
                                       ->  Bitmap Heap Scan on foreman_tasks_tasks foreman_tasks_tasks_1  (cost=4.44..12.28 rows=2 width=24) (actual time=0.848..5.291 rows=5614 loops=1)
                                             Recheck Cond: (((type)::text = 'ForemanTasks::Task::DynflowTask'::text) AND ((label)::text = 'Actions::Katello::Repository::Sync'::text))
                                             Heap Blocks: exact=1088
                                             ->  Bitmap Index Scan on index_foreman_tasks_tasks_on_type_and_label  (cost=0.00..4.43 rows=2 width=0) (actual time=0.723..0.724 rows=5614 loops=1)
                                                   Index Cond: (((type)::text = 'ForemanTasks::Task::DynflowTask'::text) AND ((label)::text = 'Actions::Katello::Repository::Sync'::text))
                                       ->  Bitmap Heap Scan on foreman_tasks_locks  (cost=33.46..45.08 rows=1 width=20) (actual time=1.231..1.231 rows=1 loops=5614)
                                             Recheck Cond: ((task_id = foreman_tasks_tasks_1.id) AND ((resource_type)::text = 'Katello::Repository'::text))
                                             Filter: (resource_id = ANY ('{59,516,517,518,520,519,8203,8241}'::integer[]))
                                             Rows Removed by Filter: 1
                                             Heap Blocks: exact=5623
                                             ->  BitmapAnd  (cost=33.46..33.46 rows=3 width=0) (actual time=1.229..1.229 rows=0 loops=5614)
                                                   ->  Bitmap Index Scan on index_foreman_tasks_locks_on_task_id  (cost=0.00..16.60 rows=558 width=0) (actual time=0.004..0.004 rows=5 loops=5614)
                                                         Index Cond: (task_id = foreman_tasks_tasks_1.id)
                                                   ->  Bitmap Index Scan on index_foreman_tasks_locks_on_resource_type_and_resource_id  (cost=0.00..16.60 rows=558 width=0) (actual time=1.236..1.236 rows=21466 loops=5558)
                                                         Index Cond: ((resource_type)::text = 'Katello::Repository'::text)
                     ->  Bitmap Heap Scan on foreman_tasks_tasks  (cost=17.76..25.60 rows=2 width=268) (actual time=3.633..3.633 rows=1 loops=8)
                           Recheck Cond: ((started_at = (max(foreman_tasks_tasks_1.started_at))) AND ((type)::text = 'ForemanTasks::Task::DynflowTask'::text))
                           Heap Blocks: exact=8
                           ->  BitmapAnd  (cost=17.76..17.76 rows=2 width=0) (actual time=3.627..3.627 rows=0 loops=8)
                                 ->  Bitmap Index Scan on index_foreman_tasks_tasks_on_started_at  (cost=0.00..6.65 rows=315 width=0) (actual time=0.003..0.003 rows=1 loops=8)
                                       Index Cond: (started_at = (max(foreman_tasks_tasks_1.started_at)))
                                 ->  Bitmap Index Scan on index_foreman_tasks_tasks_on_type  (cost=0.00..10.78 rows=315 width=0) (actual time=3.600..3.600 rows=63028 loops=8)
                                       Index Cond: ((type)::text = 'ForemanTasks::Task::DynflowTask'::text)
               ->  Index Scan using index_foreman_tasks_locks_on_task_id on foreman_tasks_locks locks_foreman_tasks_tasks  (cost=0.42..30.63 rows=558 width=20) (actual time=0.006..0.007 rows=5 loops=8)
                     Index Cond: (task_id = foreman_tasks_tasks.id)
 Planning Time: 0.699 ms
 Execution Time: 6960.680 ms
(41 rows)

Could you try running the following SQL and see if it improves the state:

CREATE INDEX temp_tasks_test ON foreman_tasks_locks (resource_type, task_id);

Afterwards you can remove it using

DROP INDEX temp_tasks_test;

That helped ALOT, output:

Unique (cost=145.34…145.49 rows=4 width=272) (actual time=62.299…62.314 rows=8 loops=1)
-> Sort (cost=145.34…145.35 rows=4 width=272) (actual time=62.299…62.299 rows=16 loops=1)
Sort Key: foreman_tasks_tasks.id, foreman_tasks_tasks.label, foreman_tasks_tasks.started_at, foreman_tasks_tasks.ended_at, foreman_tasks_tasks.state, foreman_tasks_tasks.result, foreman_tasks_tasks.external_id, foreman_tasks_tasks.parent_task_id, foreman_tasks_tasks.start_at, foreman_tasks_tasks.start_befor
e, foreman_tasks_tasks.action, foreman_tasks_tasks.state_updated_at, foreman_tasks_tasks.user_id, foreman_tasks_locks.resource_id
Sort Method: quicksort Memory: 33kB
-> Nested Loop (cost=62.62…145.30 rows=4 width=272) (actual time=37.792…62.245 rows=16 loops=1)
Join Filter: (foreman_tasks_locks.resource_id = locks_foreman_tasks_tasks.resource_id)
Rows Removed by Join Filter: 24
-> Nested Loop (cost=62.20…70.10 rows=2 width=272) (actual time=37.778…62.171 rows=8 loops=1)
-> GroupAggregate (cost=44.44…44.46 rows=1 width=12) (actual time=34.151…34.648 rows=8 loops=1)
Group Key: foreman_tasks_locks.resource_id
-> Sort (cost=44.44…44.45 rows=1 width=12) (actual time=34.078…34.312 rows=3170 loops=1)
Sort Key: foreman_tasks_locks.resource_id
Sort Method: quicksort Memory: 245kB
-> Nested Loop (cost=8.88…44.43 rows=1 width=12) (actual time=0.998…33.560 rows=3170 loops=1)
-> Bitmap Heap Scan on foreman_tasks_tasks foreman_tasks_tasks_1 (cost=4.44…12.28 rows=2 width=24) (actual time=0.803…3.458 rows=5614 loops=1)
Recheck Cond: (((type)::text = ‘ForemanTasks::Task::DynflowTask’::text) AND ((label)::text = ‘Actions::Katello::Repository::Sync’::text))
Heap Blocks: exact=1088
-> Bitmap Index Scan on index_foreman_tasks_tasks_on_type_and_label (cost=0.00…4.43 rows=2 width=0) (actual time=0.681…0.681 rows=5614 loops=1)
Index Cond: (((type)::text = ‘ForemanTasks::Task::DynflowTask’::text) AND ((label)::text = ‘Actions::Katello::Repository::Sync’::text))
-> Bitmap Heap Scan on foreman_tasks_locks (cost=4.45…16.07 rows=1 width=20) (actual time=0.004…0.004 rows=1 loops=5614)
Recheck Cond: (((resource_type)::text = ‘Katello::Repository’::text) AND (task_id = foreman_tasks_tasks_1.id))
Filter: (resource_id = ANY (’{59,516,517,518,520,519,8203,8241}’::integer))
Rows Removed by Filter: 1
Heap Blocks: exact=5623
-> Bitmap Index Scan on temp_tasks_test (cost=0.00…4.45 rows=3 width=0) (actual time=0.003…0.003 rows=2 loops=5614)
Index Cond: (((resource_type)::text = ‘Katello::Repository’::text) AND (task_id = foreman_tasks_tasks_1.id))
-> Bitmap Heap Scan on foreman_tasks_tasks (cost=17.76…25.60 rows=2 width=268) (actual time=3.437…3.437 rows=1 loops=8)
Recheck Cond: ((started_at = (max(foreman_tasks_tasks_1.started_at))) AND ((type)::text = ‘ForemanTasks::Task::DynflowTask’::text))
Heap Blocks: exact=8
-> BitmapAnd (cost=17.76…17.76 rows=2 width=0) (actual time=3.433…3.433 rows=0 loops=8)
-> Bitmap Index Scan on index_foreman_tasks_tasks_on_started_at (cost=0.00…6.65 rows=315 width=0) (actual time=0.003…0.003 rows=1 loops=8)
Index Cond: (started_at = (max(foreman_tasks_tasks_1.started_at)))
-> Bitmap Index Scan on index_foreman_tasks_tasks_on_type (cost=0.00…10.78 rows=315 width=0) (actual time=3.429…3.429 rows=63028 loops=8)
Index Cond: ((type)::text = ‘ForemanTasks::Task::DynflowTask’::text)
-> Index Scan using index_foreman_tasks_locks_on_task_id on foreman_tasks_locks locks_foreman_tasks_tasks (cost=0.42…30.63 rows=558 width=20) (actual time=0.005…0.006 rows=5 loops=8)
Index Cond: (task_id = foreman_tasks_tasks.id)
Planning Time: 0.742 ms
Execution Time: 62.399 ms
(38 rows)

1 Like

That is great! I’ve opened up a PR to add this index in the tasks plugin :slight_smile:

2 Likes

Question can i keep it on for now and can it cause any issues or any cautions i should take when i am going to upgrade to next version?

you can keep it for now and remove it when upgrading. even if you forget and it is left after the upgrade it shouldn’t cause too much issues - you’ll just have a duplicate index.

Thanks, i will keep it in mind. About the other issue that i was unable to perform “foreman_tasks:cleanup” task, should i open another ticket for this?

[root@ foreman]# foreman-rake foreman_tasks:cleanup TASK_SEARCH=‘label ~ *’ AFTER=‘1d’ VERBOSE=true
rake aborted!
The Dynflow world was not initialized yet. If your plugin uses it, make sure to call Rails.application.dynflow.require! in some initializer
/opt/theforeman/tfm/root/usr/share/gems/gems/dynflow-1.4.3/lib/dynflow/rails.rb:75:in world' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-1.1.1/lib/foreman_tasks/cleaner.rb:80:in initialize’
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-1.1.1/lib/foreman_tasks/cleaner.rb:8:in new' /opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-1.1.1/lib/foreman_tasks/cleaner.rb:8:in run’
/opt/theforeman/tfm/root/usr/share/gems/gems/foreman-tasks-1.1.1/lib/foreman_tasks/tasks/cleanup.rake:37:in block (3 levels) in <top (required)>' /opt/rh/rh-ruby25/root/usr/share/gems/gems/rake-12.3.0/exe/rake:27:in <top (required)>’
Tasks: TOP => foreman_tasks:cleanup => foreman_tasks:cleanup:run
(See full trace by running task with --trace)

That does look like an unrelated issue, is dynflow running?

A suggestion for even improving the index further was brought up on IRC, can you run

DROP INDEX temp_tasks_test;
CREATE INDEX temp_tasks_test ON foreman_tasks_locks (task_id, resource_type, resource_id);

And check if it still leads to same results or perhaps even better ones?

According to “foreman-maintain service status” all services are running. I assume dynflow

Its hard to say if its better, i would say its about the same:

CREATE INDEX temp_tasks_test ON foreman_tasks_locks (resource_type, task_id);

EXPLAIN ANALYZE speed:

Planning Time: 0.742 ms
Execution Time: 62.399 ms

CREATE INDEX temp_tasks_test ON foreman_tasks_locks (task_id, resource_type, resource_id);

EXPLAIN ANALYZE speed:

Planning Time: 0.680 ms
Execution Time: 60.201 ms

how long does the subscription page take to load now?
I would suggest opening a new thread about the task cleanup, not sure why that’s failing and it isn’t related directly to this issue.

Subscription page loads about the same or bit faster with resource_id
without resource_id

2020-06-11T13:08:21 [I|app|fa3ebe3d]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.15.1.1/app/views/katello/api/v2/subscriptions/index.json.rabl within katello/api/v2/layouts/collection (194.4ms)

with resource_id

2020-06-11T13:09:10 [I|app|5dd8a328]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.15.1.1/app/views/katello/api/v2/subscriptions/index.json.rabl within katello/api/v2/layouts/collection (145.6ms)

One of the pages i see that has more significant load time is repository view in products page. Takes about ~14seconds with 9 repositories to load(url: /products/10/repositories?page=1&per_page=40)

Is the 14 seconds a new slowness after the added index, or was that already occurring previously and is just another place we need to investigate and improve?

Or actually i just realized i was asking for the wrong page - i meant to ask about the impact on the product page, not the subscriptions page.

As far as i can remember it has always been this slow so its not anything new. This is just something that can be improved but its not a major issue as it was with this products page.

I would say that products page with or without products_id is same, tested it multiple times and the difference is 50-100ms, sometimes its in favor of products_id and sometimes its not.
with products_id

2020-06-11T14:07:50 [I|app|5706c61d]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.15.1.1/app/views/katello/api/v2/products/index.json.rabl within katello/api/v2/layouts/collection (459.6ms)

without products_id

2020-06-11T14:11:32 [I|app|849f5dd3]   Rendered /opt/theforeman/tfm/root/usr/share/gems/gems/katello-3.15.1.1/app/views/katello/api/v2/products/index.json.rabl within katello/api/v2/layouts/collection (443.6ms)
1 Like