On Wed, Mar 27, 2024 at 02:30:59PM -0700, Yuanchu Xie wrote:

Promotion/Demotion Similar to proactive reclaim, a workingset report enables demotion to a slower tier of memory. For promotion, the workingset report interfaces need to be extended to report hotness and gather hotness information from the devices[1].

[1] https://www.opencompute.org/documents/ocp-cms-hotness-tracking-requirements-...

Sysfs and Cgroup Interfaces

The interfaces are detailed in the patches that introduce them. The main idea here is we break down the workingset per-node per-memcg into time intervals (ms), e.g.

1000 anon=137368 file=24530 20000 anon=34342 file=0 30000 anon=353232 file=333608 40000 anon=407198 file=206052 9223372036854775807 anon=4925624 file=892892

I realize this does not generalize well to hotness information, but I lack the intuition for an abstraction that presents hotness in a useful way. Based on a recent proposal for move_phys_pages[2], it seems like userspace tiering software would like to move specific physical pages, instead of informing the kernel "move x number of hot pages to y device". Please advise.

[2] https://lore.kernel.org/lkml/20240319172609.332900-1-gregory.price@memverge....

Please note that this proposed interface (move_phys_pages) is very unlikely to be received upstream due to side channel concerns. Instead, it's more likely that the tiering component will expose a "promote X pages from tier A to tier B", and the kernel component would then use/consume hotness information to determine which pages to promote.

(Just as one example, there are many more realistic designs)

So if there is a way to expose workingset data to the mm/memory_tiers.c component instead of via sysfs/cgroup - that is preferable.

The 'move_phys_pages' interface is more of an experimental interface to test the effectiveness of this approach without having to plumb out the entire system. Definitely anything userland interface should not be designed to generate physical address information for consumption unless it is hard-locked behind admin caps.

Regards, Gregory