← Back to Categories

Writing

The Hybrid Framework: A Case for Hierarchical Control

The Hybrid Framework: A Case for Hierarchical Control

The current hive of development and funding around the obsession with End-to-End (E2E) control is perhaps not an evolution of intelligence. E2E is the idea that a robot should map raw pixels directly to motor torques, all the way from the vision space to the action space. This is pursued over a wholly deterministic architecture that utilizes semantic landmarks to bridge the gap between high-level perception and physical reality. In doing this, the philosophy trades transparency and a more discrete approach for superficial, "human like fluidity." This trade off comes at a very critical cost, computational parsimony. E2E often asks the model to "intuitively" learn the inverse kinematics of the manipulator geometry, something that can be computed using solvers. A multi billion parameter Vision Language Action (VLA) model, such as RT-2 or OpenVLA, can require upwards of 50 gigabytes of RAM for inference, whereas a deterministic solver using look up tables can occupy as little as 50 megabytes, while running orders of magnitude faster. Using a purely deterministic approach, on the other hand, leads to its own challenge. One of the major reasons behind the adoption of VLAs is their ability to classify and understand objects and their characteristic of object permanence, despite their problems with memorizing solutions. This is something that a inverse kinematics solver lacks. To understand why inference and efficiency are both paramount, one must look at the "Turing Test" for robotics, the problem of folding laundry. The challenge of folding laundry is widely considered as the "ultimate challenge" of domestic robotics because it involves deformable topology. A t-shirt has no fixed shape. It collapses, wrinkles, and behaves according to complex frictional and gravitational forces. An E2E model would attempt to solve this by treating the shirt as a black box of pixels. It would, through millions of training cycles, learn to associate specific visual states with a specific control for each actuator. This is a fundamentally resource intensive process. To do this whole process is unnecessary and risks overfitting. To address the limitations of a purely deterministic solver, specifically its inability to perceive state changes in deformable objects, we integrate a hierarchical "handshake", creating a hybrid system between the two. In this model, the high parameter VLA is utilized as a Semantic Perception Engine (SPE). We task the SPE to perform landmark detection. When presented with a t-shirt, it is to identify semantic anchors like the collar, the shoulder seams and the sleeves. It is to then project these anchors into a 3D task space. Once these coordinates have been established, the SPE's job is finished. It has provided the location and the context needed. The process is then handed off to a modernized inverse kinematics solver. While the SPE sets the overarching objective, a high-frequency hybrid solver manages the "local" execution. This is especially practical in the context of deformable geometry or a highly dynamic system. This would eliminate the expensive computing required by an E2E model for small shifts. The hybrid solver utilizes a higher frequency feedback loop achieved by employing a lightweight secondary vision model. This would enable the system to perform micro adjustments in real time, essentially "fine tuning" at the hardware level. This would also allow for further compensation in hardware variance. This hierarchical approach mimics the biological distinction between the cerebral cortex (high-level planning) and the cerebellum (fine motor control), ensuring that the robot remains responsive to the environment without saturating its computational bandwidth. A parallel volumetric perception layer can also be implemented to account for the surrounding geometry. This can be achieved by generating a low resolution mesh or point cloud of the action space using light weight depth sensing hardware. This can then be handed off to the IK solver to enhance its planning abilities and define out of bound areas, enhancing safety. One significant advantage of this approach is its inherent ability to generalize and the very low costs associated with it. A traditional E2E system is notorious for failing when presented with a shirt of a color or texture not found in the training set. This is because the model often conflates the "concept" of a fold with the "pixels" of a specific fabric. By decoupling perception and actuation, we introduce a Geometric Inductive Bias. The SPE is trained only to recognize semantic anchors, which makes its ability to generalize very high. Once the collar is identified as a coordinate, the IK solver treats it as a point in task space. The math required to move a robotic arm to a coordinate does not change whether the shirt is silk, cotton, or polyester. This allows the system to operate across a vast array of environments with minimal retraining. As the final layer of this framework, we can implement a compensation mesh for hardware calibration. This mesh would account for non linearities in the real world. It would function as a look up table of "deltas" mapped across the action space. This technique is already refined in high precision additive and subtractive manufacturing. During the transfer of data from the SPE to the hybrid solver, the solver queries the compensation mesh to adjust the target. This level of calibration is virtually impossible in an E2E system, where hardware imperfections are often "smudged" into the neural weights, leading to inconsistent performance if there are slight variations in hardware. This hybrid framework is also especially useful in debugging. An E2E system works like a black box, whereas decoupling the vision and actuation parts would make it easier and possible to debug the two systems independently. This system allows us to utilize the multi-billion parameter VLA for what it is best at, understanding the "what" and the "where" of the world and delegating the "how" to the cold, clinical efficiency of deterministic mathematics. The future of autonomous labor lies in this handshake: where the semantic understanding of the VLA meets the mathematical certainty of the compensation mesh and the inverse kinematics solver. You can find the pdf here. Image taken by me in Kazakhstan.

View More →
[Against] “Should holding at least a bachelor’s degree be compulsory for those contesting public elections?”

[Against] “Should holding at least a bachelor’s degree be compulsory for those contesting public elections?”

We elect politicians to be representatives and representation relies on understanding communities and their problems. The only representation a bachelor’s degree provides is the classroom it was earned in, a classroom disconnected from the realities of the ordinary people. A bachelor’s degree does not hold the ability to shape one’s integrity, leadership, or one’s connection to people. Those qualities are shaped in the real world, in conflict, in defeat and in community, not in corridors which have never tasted the struggles of the ordinary citizen. Therefore, a bachelor’s degree gives a politician nothing but the illusion of competence, and democracy cannot serve illusions. If competence cannot be created in a classroom, we must ask ourselves: what does a capable politician look like, and why do we elect them? We elect a politician to represent us, our visions, and our problems. We elect them for the burdens they are willing to carry on our behalf and for their ability to make decisions with real consequences. A capable politician is forged by the pressure of real-world crises, by the humility learned in failure, and by judgment honed through decisions that carry irreversible consequences, not by sitting in an institution and making decisions on a timed test. A capable politician possesses political acumen, a trait acquired through lived experiences and the ability to navigate intricate human realities. Such qualities cannot be certified or even adequately measured by a bachelor’s degree. As Plato, the Greek philosopher, explains in his analogy of the Ship of State, a ship cannot be navigated by those who lack the knowledge, judgment, skill, or experience to steer it. Similarly, merely possessing formal qualifications does not make one a competent captain of the nation. History backs this up. Look at Harry S. Truman, Winston Churchill, or Brazil's Lula da Silva. None of them had a standard university degree. Yet, they all steered their nations through wars and economic collapse, relying on their vision and determination, not a piece of paper. This just compounds the fact that political acumen comes from actions in the real world, not from certificates on the wall. A degree can not replicate the judgment or vision needed for true political acumen. Mandating a bachelor’s degree as a prerequisite for public office does not strengthen democracy; it dismantles it by turning governance into a gated community for the credentialed. It creates a barrier for countless capable individuals who have the skills and the experience in non academic arenas, but are not a part of the educated elite. It undermines the reality of inequality. Access to higher education, regardless of its validity towards political competence, is still determined by several external factors, opportunity and social conditions. A degree requirement would not elevate leadership, it would only filter out the people who cannot acquire a formal education and in turn, silence the voices of the ones who know the grass roots the best. This would in turn create a bigger divide, leading to a failure of the political system. In conclusion, the idea that a bachelor’s degree be compulsory for politicians is misguided at best, if not dangerous. A degree can not provide certainty for leadership or competence. History, from Lincoln to Lula da Silva shows that political acumen is built in the real world. Forcing degrees as a requirement only prioritizes privilege over ability, paper over experience and theory over the truths of the people. After all, the biggest test for a politician is the election itself; one that tests what is needed in practice, not on paper. You can find the original here. Image taken by me in Kazakhstan.

View More →
Robotics on the assembly line: Adaptability or Precision?

Robotics on the assembly line: Adaptability or Precision?

Robotics on the assembly line has always seemed like a vision of the future arrived early. We watch mechanical arms move with hypnotic precision, executing tasks with a repeatability that no human could ever match. However, upon closer examination of the modern factory floor, the glossy promise of automation often clashes with a gritty reality. The industry is realizing that while we have mastered precision, we are surprisingly poor at generalization, the task of adapting in noisy and non perfect environments. In the meticulously controlled environment, robots are flawless. However, upon encountering the chaotic and unpredictable nature of the real world, they often become fragile. This fragility is primarily attributed to what engineers refer to as the “Sim to real gap.” In a computer simulation, parts are precisely positioned, lighting is perfectly controlled, and friction remains constant. The real world, however, is filled with noise. Even minor variations in the temperature, a part’s shape or shifts in ambient light can confuse a machine that lacks human intuition. This becomes a logistical challenge during production changes. For manufacturers operating “High mix and Low volume” lines, where products frequently change, the robot’s inability to adapt means that switching tasks is not a quick adjustment; it is often a costly ordeal involving reprogramming and recalibration. As a result, we end up with a system that is exceptionally fast at performing a specific task but helpless when it needs to pivot. Furthermore, our “set it and forget it” dream overlooks the real world costs of maintenance. When a worker spots a defect, like a misaligned screw, they can instinctively decide to adjust or toss it. A robot, on the other hand, doesn’t have that intuition. It will often try to push through, forcing the part into place even when it’s not right, leading to bigger problems down the line. This lack of flexibility means more errors, more delays, and a system that’s less resilient when things go wrong. Even the popular “cobots” designed to collaborate with humans can often feel cumbersome, as they require constant supervision and lack adequate flexibility. Ultimately, until we can teach machines to navigate the complexities of reality rather than solely relying on the geometry of a simulation, we are left with tools that are remarkably precise but critically inflexible. You can find the original here. Image taken by me in Kazakhstan.

View More →
Nano Banana: The Photoshop Killer?

Nano Banana: The Photoshop Killer?

The Al boom is definitely taking the world by storm. It has only been three years, and already a lot has been accomplished. By comparison, at the same point in the mobile phone boom, the iPhone 4 hadn't even been released yet. Assumptions and sentiments about Al have become much more defined now. Enter Nano Banana, officially known as Gemini 2.5 Flash Image. Developed by Google DeepMind, this cutting-edge AI model is making waves in the image generation space. While not flawless, it stands out for its impressive capabilities. What distinguishes Nano Banana is its exceptional understanding of context and nuance. While many Al models falter with intricate scenes and detailed prompts, Nano Banana excels in rendering them with ease. It demonstrates a rare comprehension of spatial relationships and artistic intent. Additionally, it maintains character consistency across multiple prompts, ensuring that characters and objects remain consistent throughout. Remarkably, it operates at a cost-effective rate of $0.039 per image. However, it's not without its shortcomings. Occasionally, it struggles with intricate details and text, such as adding unintended letters to words or misrepresenting human hands. While Nano Banana may not render tools like Photoshop obsolete just yet, it otters a glimpse into the future of Al. It signities a shitt towards "agentic" Als: models capable of making consistent decisions and acting more like collaborators than mere tools. This evolution underscores that the Al boom is driven by tangible advancements rather than mere speculation. You can find the original here. Image taken by me in Kazakhstan.

View More →
“Should Cancel Culture Be Viewed as a Form of Accountability or Censorship?”
[Censorship]

“Should Cancel Culture Be Viewed as a Form of Accountability or Censorship?” [Censorship]

Cancel culture, in the modern world, has turned into an utterly destructive force and motive, that stifles free speech, healthy dialogue and justice. In this day and age, especially with the conversion of daily life more towards social media, cancel culture has become increasingly common. This has become a contentious social phenomenon. On the surface, cancel culture appears to hold individuals responsible for their actions and words. Yet, in practice, it just leads to disproportionate punishments. Humans have a psychological tendency to conform to those around them and form groups based on shared views and ideas. We have seen this throughout human evolution. This primitive tendency to conform to the people around them, is a grave expression in cancel culture. This often means that cancel culture is governed by “social” laws, pertaining to a particular group. This suppresses the truth due this very social conformity that exists in us humans. No one wants to be on the “wrong” side of any movement, because our human brains hate to be left out or alone, another human tendency that often hides the truth. Solomon Asch's famous conformity experiments prove that being “wrong” in a social setting lead us to doubt our very ability to reason, leading our brains to succumb to being governed by social justice, which is very perceptual. Once a survival mechanism, conformity now manifests as cancel culture, amplifying groupthink and punishing dissent. The biggest nail in the coffin for cancel culture bringing actual reform is the lack of proportionality and forgiveness in this punitive culture. This is greater amplified by the emotionless space of social media, which allows the targeted individual to be stormed, face career destruction and public humiliation, with little room to recover. Worse, this trend extends beyond public figures to everyday people, who lack the resources to navigate the fallout. This creates an effect of self doubt and danger in people wanting to share honest opinions and debates. What ultimately ends up happening is cancel culture manages to stifle artistic expression and silence voices that offer unique perspectives, just because some group of people did not like the idea. Cancel culture has been prevalent through human society for a long time. One early, yet famous example of this is the “Great Hunt” (or commonly know as the witch trials), which prosecuted 40,000 people to death without reason. It is another great example of how social conformity leads to the loss of narrative and judgement. Cancel culture, prioritizes an environment of punishment, rather than education, or understanding. It weaponizes outrage, leaving no room for empathy. Ultimately, cancel culture creates an environment of fear, one driven by subjective ideas, disharmony and hatred, rather than free speech of dialogue. Accountability is important, but it should also be developed in justice and fairness. If one hopes to build a future of justice, free speech and harmony, we must challenge cancel culture’s elusive, yet destructive tendencies, and replace them with those of justice, fairness, and proper judgement. You can find the original here . Image taken by me in Kazakhstan.

View More →