Trillium Labs will study recursive self-improvement and agent risk in the open, publishing details rivals keep hidden.