Last time we talked about what a Time Series is. This time, we are going to talk about what happens inside the “brain” of the AI. Specifically, we need to look at whether a model understands the last state (what just happened) or if it doesn’t.
In the AI world, we call this Stateful vs. Stateless thinking.
To understand this, imagine two different people watching a movie. One person has a perfect memory of the last scene, while the other person forgets everything every time they blink. Which one do you think will understand the story?
Memory Factors.
In some tasks, the AI follows a simple path:
graph LR
A[Input] --> B[Process]
B --> C[Output]
%% Styling for "Strong" Blowfish look
style A fill:#1a1a1a,stroke:#facc15,stroke-width:2px,color:#fff
style B fill:#1a1a1a,stroke:#facc15,stroke-width:2px,color:#fff
style C fill:#1a1a1a,stroke:#facc15,stroke-width:2px,color:#fff
The model looks at one thing, makes a decision, and then forgets it immediately.
However, when you want the AI to understand the context of a sequence, it needs to know what has changed. To do this, the model must track two things:
- The Initial Condition: Where did we start?
- The Difference: What is different in the current state compared to the last one?
Without this “memory,” the AI is just guessing based on a snapshot. With it, the AI can see the trend.
The Stateless Brain (No Memory)
The first type of model is Stateless. You can think of this as a “Snap Decision” model.
In this setup, every time the AI receives an input, it treats it like the first time it has ever seen anything. It doesn’t care about what happened one second ago, and it doesn’t care about what happens next. It only cares about the Input right in front of it.
Why would we want a model that forgets?
- Speed: Because it doesn’t have to “think” about the past, it is incredibly fast.
- Simplicity: It requires much less computing power.
- Specific Tasks: If you just want to know “Is there a cat in this specific frame?”, you don’t need to know where the cat was five minutes ago. You just need to see the cat now.
In our video example, a stateless model looks at 24 frames and says “Person” 24 times. It’s accurate for the image, but it’s “blind” to the movement.
The Stateful Brain (With Memory)
This is where Time Series really comes to life. A Stateful model keeps a “hidden note” for itself. When it processes Frame #2, it reads the note it wrote during Frame #1.
By comparing the “Now” with the “Before,” the AI can calculate:
- Velocity: How fast is the object moving?
- Direction: Where is it going?
- Intent: Is the person about to trip, or are they just stepping over something?
The “Memory” isn’t just a storage bin; it is the context that turns a static picture into a moving story.
The Memory Cost
If we get better results with memory, why don’t we apply it to every process? The answer is simple: No, sometimes memory actually causes a worse outcome.When your data points aren’t related to each other, you don’t need a “story.” In those cases, a Stateless model is better because it can run parallel processes. It can look at 1,000 images all at the exact same time because it doesn’t need to wait to see “Image #1” before it understands “Image #2.” This gives you a much faster result.
Memory isn’t free: it has a “price” in terms of speed and computing power.
The Selection Rule:
- Choose Stateless if you need lightning-fast speed and the “now” is all that matters.
- Choose Stateful if the “now” is useless without the “before.”
graph TD
Start[Data Observation] --> Q1{Does Order Matter?}
%% Non-Memory Path
Q1 -->|No: Independent| NM[Non-Memory / Static]
NM --> NM_Ex[Random Forest / XGBoost / SVM]
NM_Ex --> Use1[Best for: Lab Results, Demographics]
%% Memory Path
Q1 -->|Yes: Sequential| M[Memory / Temporal]
M --> Q2{Pattern Duration?}
Q2 -->|Short-term / Local| CNN[1D-CNN]
CNN --> Use2[Best for: Heart rate spikes, OCR motion]
Q2 -->|Long-term / Trend| RNN[LSTM / GRU / Transformer]
RNN --> Use3[Best for: Disease progression, 24h Trends]
%% Styling
style Start fill:#1a1a1a,stroke:#facc15,stroke-width:2px,color:#fff
style NM fill:#2d3748,stroke:#facc15,stroke-width:1px,color:#fff
style M fill:#2d3748,stroke:#facc15,stroke-width:1px,color:#fff
Some models carry short memory, some long memory, and some simulate memory without truly having it. In Part III, we’ll explore how different architectures fake or manage ‘memory’, and why that matters for performance and cost.