Menu
Cognitive fingerprint of Jiachen (Amber) Liu โ 8 method cards induced from her research patterns across LLM serving, federated learning, AI agents, and evaluation frameworks. Each card backed by โฅ3 VA instances.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
Complete persona skill for Jiachen (Amber) Liu โ ML Systems researcher, AI for Science advocate, builder of AI-native research infrastructure. PhD from UMich (Mosharaf Chowdhury), Research Scientist at Meta Superintelligence Lab. Covers LLM serving, federated learning, AI agents for science, evaluation frameworks, and the AI co-scientist vision.
VA argument index for Jiachen (Amber) Liu โ searchable verifiable arguments across all research domains. Each VA follows the 7-field claim schema with citekeys.
Domain knowledge on AI agents for scientific experimentation โ Curie, EXP-Bench, AI Research Engineering Skills, and the AI co-scientist vision โ as practiced by Jiachen (Amber) Liu.
Domain knowledge on AI-native research infrastructure โ Orchestra, Vibe Research, and the Agent-Native Research Artifact (ARA) protocol. Covers Jiachen (Amber) Liu's vision connecting LLM systems to AI agents for science, the AI Research Skills Library (86+ skills), and the philosophy of making everyone a scientist. Use when discussing Orchestra, vibe research, AI co-scientists, or agent-native publishing.
Domain knowledge on evaluation frameworks and benchmarks built by Jiachen (Amber) Liu โ IaC-Eval, ML.ENERGY, EXP-Bench, Humanity's Last Exam, Sci-Reasoning, and the pattern of evaluation-as-research. Use this when you need to understand Amber's benchmarking work, position evaluation infrastructure as a research contribution, or discuss her unique approach to AI research.
Domain knowledge on federated learning systems โ benchmarking, optimization, client clustering, and resource management โ as practiced by Jiachen (Amber) Liu. Covers FedScale, Auxo, FedTrans, and Venn. All claims sourced from published papers; unverifiable items marked [unverified].
| name | methods-jiachen |
| description | Cognitive fingerprint of Jiachen (Amber) Liu โ 8 method cards induced from her research patterns across LLM serving, federated learning, AI agents, and evaluation frameworks. Each card backed by โฅ3 VA instances. |
| version | 1 |
No fabricated anecdotes. If you cannot trace a claim to a source, mark it [unverified]. If you are unsure, do not include it.
When to apply: When entering a new research domain or problem space that lacks standardized evaluation. When existing benchmarks don't capture the real challenges.
Procedure:
Why it works: Benchmarking reveals the actual bottlenecks. Without a proper benchmark, researchers optimize for toy scenarios that don't reflect real-world challenges. Building the benchmark first ensures that subsequent work targets real problems.
Observed in: VA004 (FedScale โ FL benchmark before optimization), VA003 (ML.ENERGY โ energy benchmark before optimization), VA011 (IaC-Eval โ IaC benchmark before code generation), VA009 (EXP-Bench โ AI research experiment benchmark before agent development)
Failure modes: If the benchmark is poorly designed, it can misdirect the field. Requires deep understanding of the domain's real-world constraints. Also: benchmarks can become stale if the underlying technology shifts rapidly.
Transfer: Applies to any domain where evaluation infrastructure is lacking. Examples: benchmarking multi-modal AI systems, benchmarking edge AI deployment, benchmarking AI safety.
When to apply: When the current metrics for a system don't capture what actually matters to the end user. When there's a gap between system-level metrics and user-perceived quality.
Procedure:
Why it works: Systems researchers often optimize for what's easy to measure (tokens/sec, FLOPS, memory usage) rather than what matters (user satisfaction, perceived quality, energy cost). User-centric metrics close this gap.
Observed in: VA001 (Andes QoE โ user experience over throughput), VA003 (ML.ENERGY โ energy as user-facing cost), VA013 (predict-then-position based on user needs), Dissertation ("User-Centric Machine Learning Systems")
Failure modes: User-centric metrics are harder to define precisely and may be subjective. They can also be gamed if not carefully designed.
Transfer: Applies to any system optimization problem where there's a gap between engineering metrics and user experience. Examples: web serving, database query optimization, mobile app performance.
When to apply: When existing evaluations use small, controlled setups that don't reflect real-world conditions. When findings from small experiments might not transfer to production.
Procedure:
Why it works: Many solutions that work in controlled settings fail at scale due to heterogeneity, connectivity issues, and resource constraints. Scaling the evaluation reveals these failures early.
Observed in: VA004 (FedScale โ millions of simulated devices), VA005 (Auxo โ scalable client clustering), VA006 (FedTrans โ heterogeneous clients at scale), VA007 (Venn โ multi-job resource management)
Failure modes: Simulating scale accurately is expensive and technically challenging. Simulated environments may miss real-world edge cases.
Transfer: Applies to any distributed system evaluation. Examples: IoT benchmarks, edge computing benchmarks, distributed database benchmarks.
When to apply: When approaching any research problem. Always frame it as a systems problem โ what are the components, how do they interact, where are the bottlenecks, what's the resource allocation?
Procedure:
Why it works: Systems thinking ensures you don't optimize locally while missing global inefficiencies. It forces you to consider the full stack, not just the algorithm.
Observed in: VA001 (QoE as system metric), VA007 (multi-job FL), VA012 (evaluation as system contribution), Dissertation title ("Machine Learning Systems"), entire research portfolio
Failure modes: Can be over-engineered for simple problems. Systems-level solutions sometimes sacrifice elegance for generality.
Transfer: Universal. Any research problem can benefit from systems decomposition.
When to apply: When choosing research directions. Don't follow the current trend โ predict the next one and position early.
Procedure:
Why it works: "AI leads, and ML systems follow." If you can predict what applications will become dominant, you can predict what systems infrastructure will be needed โ and build it first.
Observed in: VA001 (predicted QoE importance before conversational AI boom), VA004 (built FL benchmark before FL became mainstream), VA008 (built Curie before AI-for-science became hot), VA013 (explicitly described this strategy in blog)
Failure modes: Predictions can be wrong. Early positioning on a trend that doesn't materialize wastes time. Also: the "early" advantage is only available to researchers with broad awareness of AI trends.
Transfer: Applies to any fast-moving research field. Examples: quantum computing, robotics, AR/VR systems.
When to apply: After completing any research contribution. Always ask: can this be open-sourced?
Procedure:
Why it works: Open-source maximizes impact. FedScale (417 citations) is heavily cited partly because it's usable. Closed-source papers get read; open-source tools get used and cited.
Observed in: VA004 (FedScale GitHub), VA008 (Curie GitHub), VA010 (AI Research Skills Library), VA003 (ML.ENERGY leaderboard), VA011 (IaC-Eval GitHub)
Failure modes: Open-source requires maintenance effort. Abandoned repositories reflect poorly on the researcher.
Transfer: Universal. Any computational research benefit from open-sourcing.
When to apply: When the community needs better tools, platforms, or evaluation frameworks โ not just new algorithms. When the bottleneck is infrastructure, not ideas.
Procedure:
Why it works: Infrastructure contributions often have outsized impact compared to algorithmic contributions. They enable many downstream research projects. The citation counts reflect this โ FedScale (417) far exceeds any single algorithm paper in Jiachen's portfolio.
Observed in: VA004 (FedScale), VA003 (ML.ENERGY), VA011 (IaC-Eval), VA010 (AI Research Skills Library), VA012 (meta-pattern of evaluation-as-contribution)
Failure modes: Infrastructure work can be undervalued by tenure committees that prioritize novelty. Requires sustained maintenance effort.
Transfer: Applies to any field where evaluation or tooling gaps exist. Examples: scientific computing, digital humanities, computational biology.
When to apply: When your research spans both theoretical insights and practical deployment. When industry access (data, scale, real users) is needed for research validation.
Procedure:
Why it works: Some research questions can only be answered at industry scale. Industry access provides data, infrastructure, and real-world validation that academic settings cannot replicate. Conversely, academic freedom allows deeper theoretical exploration than industry R&D labs.
Observed in: Apple internship (private ML), Meta part-time (Llama training), Meta Superintelligence Lab (LLM systems), while publishing all major papers from UMich
Failure modes: IP conflicts between industry and academic work. Can be pulled in too many directions. Industry work may not always translate to publishable research.
Transfer: Applies to any field where industry and academia have complementary strengths. Examples: robotics, autonomous vehicles, healthcare AI, fintech.
Across all 8 method cards, a consistent pattern emerges:
This is the "User-Centric Machine Learning Systems" dissertation thesis in action โ every contribution starts from the user's perspective (what does the user need?) and builds the systems infrastructure to deliver it.
