Fast/Slow Thinking For Robotics And Time Series

Thesis

Fast/slow thinking is useful in AI systems only when it stops being a loose metaphor and becomes an interface contract. The engineering question is not whether one model can “think fast” and “think slow.” The question is which layer is responsible for raw signal handling, which layer maintains latent state, which layer chooses actions or control inputs, and which layer reasons over goals, constraints, and possible futures.

Robotics is currently the cleanest public example of this shift. Recent robot policies increasingly separate slow task and constraint context from fast motor control. A vision-language model, embodied-reasoning model, symbolic planner, constraint system, latent world model, or energy-based scorer can interpret the task, scene, constraints, and possible futures; a lower-level action expert turns that context plus recent observations into continuous action chunks; an even lower controller may track joint targets or actuator commands at high frequency. GR00T N1, π0, π0.7, Helix, Helix 02, and Gemini Robotics 1.5 all point toward this layered design, even though they instantiate it differently. EBT adds the energy-based version of the slow layer: score candidate predictions with a learned compatibility function, then spend more compute only when optimization or self-verification is useful.

The same idea should shape time-series foundation models. Most current time-series models are still passive dynamics models: they predict future observations from historical observations. A real-world system, however, does not only observe. It acts. In this wiki’s terminology, the next step is latent-state time-series modeling that can support action-conditioned world models: models that maintain useful state and reason about futures under candidate actions, control inputs, or interventions. Terminology is the contract for those distinctions.

The Stack Pattern

The recurring shape is:

high-volume signal stream
  -> fast local filtering or control
  -> compressed state or short-horizon trajectory
  -> action-conditioned latent dynamics
  -> slow reasoning, constraints, and policy

The lower the layer, the higher the data rate and the stricter the latency budget. The higher the layer, the smaller the data stream and the richer the state abstraction. Intelligence does not grow by pushing a giant reasoning model down to the raw signal. It grows by preserving the right state variables as information moves upward.

This is why “just use a bigger Transformer” is the wrong default for physical or operational systems. A monolithic sequence model has to satisfy incompatible demands: raw throughput, low latency, continuous geometry, uncertainty, context, planning, safety, and explanation. Layering lets each model solve the problem it is shaped for.

Canonical Interfaces Are Where Transfer Happens

The transfer boundary is usually not the raw signal. A general robot policy should not have to reason directly in the joint coordinates, motor currents, controller gains, and actuator limits of every embodiment. It should operate through a more stable action/state contract, while a robot-specific adapter or controller maps that contract to the physical body.

In robotics, this contract may include normalized proprioception, end-effector delta pose, gripper commands, action chunks, control frequency, and embodiment metadata. Open X-Embodiment is the coarse version of this pattern: many robot datasets are aligned through selected camera streams and normalized end-effector action representations. Newer systems add richer embodiment and control metadata instead of pretending that all robot bodies have identical dynamics.

general policy or world model
  -> canonical state and action interface
  -> embodiment-specific adapter or controller
  -> joints, actuators, and safety limits

The SRE analog is direct: a production system’s “embodiment” is its service graph, telemetry schema, deployment machinery, and intervention capabilities. A transferable operational world model should not depend on fixed metric positions such as “channel 17 is service A latency.” It should condition on the system graph and expose typed intervention primitives such as rollback, scale, traffic shift, feature-flag change, circuit-breaker update, or rate-limit change.

system graph + telemetry schema + capabilities
  -> graph-conditioned latent state
  -> canonical intervention or control-input primitives
  -> system-specific executor

This is the same fast/slow principle in a different body: shared reasoning and latent dynamics above, local execution mechanics below.

Robotics: Context Slowness, Motor Speed

Robot data is not a plain forecasting table. It is an action-conditioned multimodal trajectory: camera observations, proprioception, force or tactile signals, language context, embodiment metadata, and control inputs over time. The common policy interface is:

context, observation_history, optional action_history -> action_chunk

A robot action model usually does not need to write a paragraph. It needs to produce a short, smooth, physically plausible sequence of control inputs. This is why modern systems increasingly move low-level action generation away from ordinary next-token language modeling and toward chunked continuous control.

There are several versions of this pattern:

• Diffusion Policy, RDT-1B, π0, GR00T N1, and π0.7 use diffusion or flow matching over future action chunks.
• FAST keeps an autoregressive token path but first compresses action chunks in frequency space, which is a specialized bridge between language-model training and continuous control.
• Helix uses a slow VLM semantic latent to condition a fast visuomotor Transformer for continuous upper-body actions.
• Helix 02 adds another level: System 2 handles semantic context, System 1 emits full-body joint targets, and System 0 tracks actuator-level whole-body control.
• Gemini Robotics 1.5 makes the slow layer explicit through embodied reasoning, thinking, planning, progress estimation, and subtask handoff to a VLA/action model.
• VL-JEPA is not a motor policy, but it shows how the upper perception/context layer can predict continuous target embeddings and decode language only when a text readout is needed.

The important point is not that diffusion or flow matching replaces Transformers. Transformers are still everywhere: VLM backbones, diffusion Transformers, action experts, and token policies. The real shift is that the output interface changes from text-like tokens to future control-input trajectories.

Why The Fast Layer Is Often Diffusion Or Flow

Fast does not mean simple in the mathematical sense. It means local, low-latency, and tightly coupled to the current state. For robot control, the fast layer has to solve a hard distributional problem:

p(action_chunk | observation_history, state, context)

The distribution is multimodal. The robot may grasp an object from the left or right, push before grasping, reorient the wrist, or choose a different contact sequence. A direct regression head tends to average incompatible modes. A categorical action-token model discretizes continuous geometry and can become awkward as action dimensionality rises. Diffusion and flow matching instead model a distribution over continuous chunks.

This also makes the control loop look like learned receding-horizon control:

observe
generate action chunk
execute part of the chunk
observe again
regenerate

The model does not need to solve a whole task open-loop. It repeatedly produces a local plan under fresh observations. That is why action chunking can amortize denoising or flow-integration cost while remaining responsive to perturbations.

Why The Slow Layer Still Matters

A fast action expert can be physically fluent and still semantically blind. It needs context: what object matters, what task is active, which strategy is desired, what constraints apply, and when the goal has changed.

Slow layers can provide:

• human-language grounding and task interpretation;
• typed schemas, symbolic constraints, programs, temporal logic, or task DSLs;
• latent-state and world-model predictions over possible futures;
• energy-based compatibility scores over states, actions, goals, and constraints;
• planning over subtasks and progress;
• goal and safety constraints;
• tool use or external information;
• strategy metadata such as speed, quality, control mode, and allowed mistakes;
• visual or latent subgoals supplied by a world model.

π0.7 is especially useful here because it treats prompt context as more than a task label. The prompt can include subtask text, episode metadata, control mode, and generated subgoal images. This turns “text conditioning” into a steering interface for heterogeneous trajectory data.

Generated visual subgoals are a useful bridge between VLA policies and world-model-style prediction, but they are not yet full candidate-action rollout. A subgoal image specifies a desired near-future observation for the action expert; it does not by itself compare the future trajectories that would follow from alternative action or control-input sequences. Genie is the complementary boundary case because it learns latent actions for controllable visual futures, while World Model for Robot Learning Survey and Reconstruction Or Semantics? sharpen the criterion: a predictive representation is useful for planning only when it preserves action-relevant state, not merely visually plausible frames.

Human language is only one slow-layer representation. It is often the right interface for instructions, explanations, incident narratives, and collaboration with operators. But internal semantics do not have to be written in natural language. A constraint may be better represented as symbolic logic, a typed program, a graph, a verifier, a learned latent state, a value function, or an energy function. This is closer to the autonomous-intelligence framing behind Energy-Based Models and World Models: meaning can be expressed as compatibility, reachability, controllability, or low-energy futures, not only as words.

EBT makes this concrete for a Transformer-like system. It does not use extra tokens as the only slow-thinking substrate; it optimizes the candidate prediction itself under a learned energy. For time-series or robotics systems, the useful transfer hypothesis is not “replace the whole stack with EBT.” It is narrower: use an energy-scored layer to evaluate candidate futures, candidate action chunks, or candidate interventions when a cheap fast layer is uncertain.

VL-JEPA makes this boundary concrete for vision-language systems. It predicts a target text embedding from visual input and a query, then uses a lightweight decoder only when a human-readable answer is needed. In a robot stack, this belongs above the fast motor layer: it can maintain an always-on task, scene, or progress embedding stream that conditions planners, monitors, or operator interfaces, while diffusion, flow, tokenized-action, or controller layers still handle continuous action generation.

image or video stream + query
  -> visual embeddings
  -> predicted target embedding
  -> selective language readout
  -> planner, monitor, or operator interface

That matters because language decoding becomes a sparse readout, not the default internal representation. A system can monitor changes in continuous embedding space and decode only when there is a significant semantic shift or when a human-facing explanation is required. The time-series analog is an incident, regime, or system-state embedding stream that is decoded into an operator explanation only when the state changes, confidence drops, or an intervention decision must be surfaced.

The caveat is equally important: VL-JEPA is world-model-adjacent, not a complete action-conditioned world model by itself. It predicts target or answer embeddings, not future state distributions under candidate actions, control inputs, or interventions.

The slow layer should not be asked to output every motor command. The fast layer should not be asked to decide the whole mission. The boundary is where engineering taste matters.

Telecom And Wireless: The Oldest Version Of The Pattern

Telecom and wireless systems have lived with this hierarchy for decades. At the bottom of a radio stack, the system processes an enormous stream of information under brutal latency constraints. The algorithms are intentionally simple, local, and highly optimized: synchronization, filtering, FFT/IFFT, channel estimation, equalization, coding and decoding, hybrid ARQ, beam tracking, and other physical-layer operations.

As the stack moves upward, the raw stream is compressed:

RF samples / IQ streams
  -> symbols and frames
  -> packets and flows
  -> sessions and users
  -> network state, policies, and business goals

Each step reduces bandwidth and increases abstraction. The physical layer cannot afford reflective reasoning over business intent. It must turn noisy waveforms into usable symbols quickly and predictably. Higher layers operate on smaller objects: packets, scheduling requests, channel-quality summaries, sessions, topology, KPIs, alarms, and policies. There the system can make more intelligent decisions about scheduling, handover, congestion, power, beam/resource allocation, anomaly response, capacity planning, and remediation.

The lesson for AI systems is direct: intelligence should often be placed where the representation has already been compressed into the right state variables. Putting a slow reasoning model at the raw-sample layer is usually wasteful. Putting only simple filters at the policy layer is underpowered.

Cross-Domain Mapping

Layer	Robotics	Wireless / Telecom	Time-Series AI
Raw stream	Cameras, tactile, proprioception, motor sensors	IQ samples, OFDM symbols, channel estimates	Metrics, traces, logs, event streams
Fast local layer	S0 controller, servo loop, tactile reflex, joint tracking	PHY processing, coding, synchronization, equalization	Filters, local detectors, fast anomaly gates
Short-horizon action layer	S1 action expert, diffusion/flow action chunks, visuomotor policy	MAC scheduling, beam/resource control, retransmission decisions	Short-horizon latent dynamics, local control policy
State and world layer	Action-conditioned latent/video world model, target-embedding stream	Network state, topology, session and mobility state	Latent-state model, regime model, incident embedding stream, action-conditioned world model
Slow reasoning layer	VLM planner, symbolic constraints, latent or energy scorer, task/subtask policy, selective text readout	RRC, SON, network planning, policy optimization	Incident reasoning, intervention planning, operator copilot, constraint solver, selective explanation
Outcome	Task success, safety, dexterity	Throughput, latency, reliability, coverage	Forecast utility, intervention quality, operational outcome

The analogy should not be overread. A wireless PHY stack is not a robot policy. But the system-design lesson is robust: low layers process high-volume data with tight latency; high layers reason over compressed state and make slower decisions with broader context.

Time Series: From Forecasting To Acting

Many time-series foundation models still stop at:

history -> future observations

That is useful, but it is not enough for real-world systems where actions matter. Production systems have deployments, rollbacks, autoscaling changes, traffic routing, feature flags, remediations, and incident-response decisions. Healthcare has treatments, procedures, medication doses, and device settings. Recommender systems have exposures and recommendations. Education systems have hints, items, and interventions.

The target interface is closer to:

latent_state, context, candidate_actions -> future trajectory distribution

This requires separating observations from actions, control inputs, interventions, events, and exogenous variables. If a model cannot tell a passive traffic spike from an operator rollback, it cannot be a serious action-conditioned world model.

The fast/slow split helps define the architecture:

• Fast layer: ingest high-volume observations, maintain local summaries, detect immediate anomalies, update short-horizon state.
• Middle layer: maintain latent state, model next-state dynamics, forecast plausible futures, evaluate candidate control inputs.
• Slow layer: reason over goals, constraints, operator intent, policy, safety, cost, and long-horizon consequences.

This mirrors the robotics split but changes the domain objects. Instead of motor control, the output may be an intervention recommendation, a remediation plan, a traffic-shaping choice, a treatment setting, or an experiment policy.

Design Principles

1. Do not collapse observation, state, and action. A metric, an alert, a deployment, and a rollback are not the same kind of variable.
2. Compress upward, not blindly. Lower layers should preserve variables that matter for future action consequences, not only variables that reconstruct the raw signal.
3. Keep physical or operational time explicit. Control frequency, sensor latency, sampling rate, patch duration, and action horizon are part of the model contract.
4. Use continuous interfaces where geometry matters. Numeric control inputs and physical trajectories should not be forced into ordinary language tokens without a reason.
5. Do not equate semantics with natural language. Use human language for human-facing instruction, explanation, and collaboration. Use structured symbols, typed state, programs, logic, latent representations, or energy functions when internal semantics must be precise, compositional, optimizable, or grounded in action consequences.
6. Make the embodiment or system descriptor explicit. Transfer requires metadata about robot body, control mode, service graph, telemetry schema, available interventions, and execution limits.
7. Evaluate closed-loop outcomes. Forecast error is not enough when the system is used to choose actions.
8. Make the hierarchy inspectable. Operators should know which layer chose a strategy, which layer generated a local action, and which state variables drove the decision.
9. Decode language only when language is the interface. Maintain internal state in embeddings, typed variables, symbols, constraints, or energy scores when those are better computational substrates; decode to text when an operator, log, report, or external language interface needs it.

Failure Modes

The layered design can fail in predictable ways.

• The slow layer can hallucinate goals or constraints that the fast layer cannot execute.
• A natural-language representation can be too ambiguous for constraints that should have been symbolic, typed, verified, or energy-scored.
• A system can waste latency by autoregressively decoding stable states instead of monitoring latent shifts and decoding only when a readout is needed.
• The fast layer can produce smooth local actions that satisfy the wrong task or constraint.
• A latent state can preserve easy-to-predict nuisance variables while losing action-relevant state.
• A system can optimize passive forecast error while becoming useless for intervention choice.
• A hierarchy can hide responsibility: every layer looks locally reasonable, but the composed policy fails.

These are evaluation problems as much as architecture problems. The test should ask not only “was the next observation predicted?” but “did the system choose or rank better actions under uncertainty?”

What To Build Next

The next generation of time-series systems should borrow from both robotics and telecom:

raw observations and event streams
  -> fast state compression
  -> latent dynamics and uncertainty
  -> action-conditioned rollout
  -> slow reasoning over goals and constraints
  -> selected intervention or control input

The research question is not whether to use an LLM, a Transformer, a diffusion model, an SSM, or a controller. The research question is where each belongs in the hierarchy.

Robotics suggests that fast action generation benefits from continuous chunked outputs. Telecom suggests that bottom layers should remain simple, predictable, and latency-aware. Time-series world modeling suggests that the middle layer must maintain useful latent state, not merely forecast observations. The combined lesson is that intelligence emerges from the right division of labor: raw throughput below, stateful dynamics in the middle, and goal, constraint, and policy reasoning above.

Relation To Foundation TSFM Agenda

This page is the main system-architecture bridge from the Foundation Time-Series Model Research Agenda to robotics, telecom, observability, and digital-world robots. It is not a benchmark page; its role is to keep the agenda’s slots tied to interface contracts: high-rate observations, maintained latent state, context, candidate actions, and future trajectory distributions.

The strongest agenda link is the practical north star: a foundation time-series model should become the state and dynamics layer underneath an agent that can observe, remember, simulate, and act. The missing evidence remains closed-loop evaluation in digital operational systems rather than only robotics analogy or passive forecasting.

Open Questions

• What is the time-series equivalent of a robot action chunk: a block of control inputs, a remediation plan, a treatment schedule, or a latent rollout?
• Which layers should be trained end-to-end, and which should be separated by hard engineering contracts?
• How can a slow reasoning layer inspect or constrain a fast continuous action model without destroying latency?
• What benchmarks can evaluate intervention choice rather than passive forecast accuracy?
• What is the SRE equivalent of a robot canonical action interface: rollback, scale, traffic-shift, circuit-breaker primitives, or a richer typed intervention DSL?
• Which telecom control problems should be ingested as source anchors for this analogy: scheduling, handover, power control, self-organizing networks, or incident remediation?

Related Pages

• Foundation Time-Series Model Research Agenda
• Robotics Time-Series Modeling
• Robotics Text Conditioning
• Latent-State Time-Series Modeling
• Time-Series Foundation Models
• World Models
• Observability Time Series
• Action-Conditioned Time-Series Datasets
• Energy-Based Models
• Terminology