launchd.agents.<name>.serviceConfig.HardResourceLimits.ResidentSetSize

The maximum size (in bytes) to which a process's resident set size may grow. This imposes a limit on the amount of physical memory to be given to a process; if memory is tight, the system will prefer to take memory from processes that are exceeding their declared resident set size.