ON TREND
Will richer IoT devices take intelligence closer to the edge?
SEPTEMBER 2026
COMMENT
Is on-device intelligence a classic no-brainer?
Let’s move our minds into the thought processes of a typical IoT enterprise. Their background is in making their core product or providing a specific service and they’re really good at doing that. Then, over the last two decades, IoT has emerged providing an opportunity to serve customers better, generate more revenue and compete more effectively by connecting that product or service.
The downside was IoT enablement often came with an unclear cost in terms of connectivity and cloud compute, an opaque business case — would the customers come if you built it? — and significant skills gaps to address. If you make chainsaws or washing machines, you’re not an expert in permanent roaming in Brazil or identifying where you need to site compute resources in cloud facilities to support global deployments.
Over time, many of these wrinkles have been ironed out. An industry of IoT platform providers has emerged to provide the support enterprises need and a comfortable ecosystem in which companies could do what they want for themselves or buy in technology and services from others for things they can’t or don’t want to do.
Bigger deployments, larger costs
As IoT has scaled up – it’s taken longer than expected but the billions of connected devices are now here – the costs of connectivity and cloud have risen to the extent they’re threatening the sustainability of some IoT business cases. At the same time, component costs have become more affordable and device design skills have increased. We’re at an inflection point where, for some use cases, greater on-device intelligence is cost-effective and is embedded to add features and help the enterprise keep control of their devices.
On the flipside, adding on-device intelligence means greater upfront costs but far lower service charges. Some can digest these, others can’t, but richer devices open up options for enterprises and device designers. These have come just in time as AI emerges onto the scene with AIoT components poised to be embedded in more devices than ever before.
Just like with connectivity and cloud compute, AI comes at a cost. IoT enterprises can decide to invest in greater on-device capabilities or accept higher and higher fees for centralised AI processing. In reality, both models will be adopted for the next generation of IoT and embedding AI capabilities onto devices alongside the intelligent edge and cloud AI really is a no-brainer.
INTERVIEW
"The next phase of IoT will be defined by distributed intelligence, where more decisions are made locally at the edge instead of centrally in the cloud. Devices will become more context-aware, more autonomous and more selective about what data they share.”
François de Rochebouet
Head of AI Solution Marketing, STMicroelectronics
IoT Now: Are you seeing a new focus on IoT hardware? Is what can be done on the device becoming a bigger priority as organisations seek to minimise network traffic and cloud and AI expenses?
François de Rochebouet: Absolutely. We are seeing a strong shift towards more intelligence on the device itself. Edge AI allows companies to process data locally and send only meaningful results, rather than streaming raw data to the cloud. That reduces bandwidth, lowers cloud storage and inference costs, and also improves latency, privacy and scalability. For example, a smart doorbell running a people-detection model on an STM32N6 can analyse video locally and report only relevant events, which makes the system both more efficient and more cost-effective as deployments scale.
IoT Now: How are device designers adopting new approaches and components to minimise bill of materials (BOM) costs, streamline development and simplify overall device design?
FdR: Designers are looking for highly integrated platforms that let them consolidate functions and avoid overdesign. Instead of adding separate processors or connectivity devices for every feature, they want microcontroller units (MCUs) that can combine control, sensing, security and AI inference in a single, efficient architecture. They are also adopting software tools and pre-optimised AI model pipelines that reduce development effort and help teams move faster from prototype to production. The goal is not only a lower BOM, but also a simpler board design, lower power consumption and a shorter design cycle.
IoT Now: What types of components do you see as the key enablers here?
FdR: The key enablers are efficient microcontrollers with built-in AI optimisation, secure connectivity, integrated memory strategies and strong software ecosystems. In many IoT applications, the MCU remains the critical component because it sits at the intersection of cost, power efficiency and real-time control. Beyond silicon, software is equally important: model optimisation tools, embedded AI libraries and deployment environments are what make advanced on-device intelligence practical for a broad range of developers.
IoT Now: There’s a clear hierarchy in cost and complexity extending across system-on-chips, system-on-modules and single board computers (SoCs, SoMs and SBCs). How important are these components in enabling engineers to design and manufacture more intelligent devices in less time?
FdR: They are all important, but they serve different needs. SBCs and SoMs are very useful when teams want to accelerate prototyping or handle higher-end workloads with less hardware effort upfront. But for high-volume, power-sensitive and cost-sensitive products, MCUs are often the best fit because they allow tighter optimisation of BOM, energy use and long-term manufacturability. In practice, engineers increasingly want a migration path: they may explore quickly on a more complex platform but productise on a more efficient embedded architecture once requirements are clear.
IoT Now: What comes next? Is the next era of IoT composed of things with greater embedded intelligence that are able to do more with less centralised processing and decision-making?
FdR: Yes, that is exactly the direction of travel. The next phase of IoT will be defined by distributed intelligence, where more decisions are made locally at the edge instead of centrally in the cloud. Devices will become more context-aware, more autonomous and more selective about what data they share. Cloud will still play a major role for fleet management, model updates and large-scale analytics, but more of the immediate sensing, understanding and decision-making will happen directly on the device. That shift is essential to make IoT systems more scalable, responsive and energy-efficient.
IoT Now: What do you see as the main use cases in which greater on-device intelligence is being deployed?
FdR: We see strong momentum in applications such as vision-based detection, predictive maintenance, audio event recognition, human presence detection, smart home automation, industrial monitoring and wearable or medical sensing. What these use cases have in common is the need for fast, local interpretation of sensor data without constant cloud dependence. In many cases, on-device AI is not about replacing the cloud entirely; it is about making devices smart enough to filter, classify and react immediately, while sending only the most valuable information upstream.
MARKET REPORT
Smarter devices pave the way for the advanced AIoT era
IoT used to be all about the connections to the cloud but on-device AI and greater capability at the edge is resulting in a new generation of richer, more complex devices. These cost more to manufacture and can take longer to develop but embedded features can help control network and cloud expenses by enabling more processing to happen on the device with lower latency. The flipside is long-life deployment scenarios can be hindered if it’s not practical to upgrade features during device lifespans, although software and firmware updates are well understood across IoT.
This leaves device designers and developers looking to cut reliance on centralised intelligence in the cloud because of its costs, latency and potentially unsustainable remote AI workloads. Enterprises are looking to evade these by adding greater functionality to the devices themselves – provided the numbers can be made to stack up. Scott Brenton, the executive director of Solsta Embedded, agrees that this cost-driven focus is affecting device design. “Customers are asking or considering what data needs to leave the device at all, not just how to process it once it’s in the cloud,” he explains. “This is not so much a new focus but, rather, a different view on what to use.”
SOM things stirring up IoT innovation
The focus is on maximising value and finding the right balance between costs, flexibility and time to market. “Customers are choosing standard over custom,” confirms Brenton. “Nobody wants the lead time or risk of full custom when a standard system-on-module (SOM) gets them there faster. A SOM can be standard in terms of the operating system being used, such as open source Linux.”
“The key enabling components are higher power, more capable SOMs available at lower price points,” he points out. “It’s about balancing cost versus control versus time-to-market. Most are trading bill of materials (BOM) cost for speeds, especially with lead times under intense pressure.”
That balance has to be achieved in a market that is rapidly transforming from relative dumb devices that depended on centralised analysis in the cloud to a lower-latency AI-powered environment of on-device intelligence. “IoT – specifically industrial IoT – began as relatively cheap, simple sensors connected to the cloud to handle the processing demand,” explains Jake Kochnowicz, the chief business officer at Imagination Technologies. “In a short period, that evolved into high-performance edge AI that draws on large amounts of data from ever-more complex devices. We’re seeing intelligence moving to the edge for several reasons: real-time, deterministic response that’s possible with no latency from communications; operations can continue if connectivity fails, providing unparalleled reliability; and data is kept in-house, offering greater privacy. Additionally, cloud services are expensive whereas local processing could reduce a company’s expenditure.”
99% of devices lack true edge AI
Even so, in its State of Enterprise IoT 2026 report, IoT Analytics estimates that, as of December 2025, less than 1% of the 21.1 billion IoT connections had a true edge AI component. The firm describes these as components comprising dedicated AI accelerators like GPUs or NPUs. The under 1% figure underscores how little intelligence there really is in IoT devices today. But IoT Analytics expects this share to shoot up massively in the coming years as the industry enters the next wave with agentic and physical AI.
That suggests there’s massive appetite to add AI and other on-device intelligent capabilities but achieving that level of adoption won’t be easy. “There are serious headwinds,” acknowledges Kochnowicz. “To build such rich capabilities, engineers rely on specific silicon components. The three main technologies are general purpose graphics processing units (GPUs): processors that handle graphics, processing, and parallel compute workloads simultaneously, they are key accelerators for compute and AI tasks; neural processing units (NPUs): the dedicated silicon optimised for low-power, inference AI use on the device; and central processing units (CPUs): every chip needs one for control and they have increasing AI capabilities.”
Brenton has no doubt the shift is already underway. “There will be more local and edge decision-making and less ‘send everything to the cloud.’ Reliable networks are likely to become more critical,” he adds. “Take 4G/5G as an example – cellular is positioned as a best-effort service with no service level agreement (SLA). This great coverage is open to a huge range of applications, but you don’t control the network so reliability and consistency are never guaranteed. Private networks may be the next consideration for users who need more control.”
Devices ditch the datacentre
In its 2026 report, IoT Analytics also confirms that chipmakers are shifting IoT intelligence to the edge. The new focus on autonomous, agentic operations requires real-time decision-making which cloud architectures often cannot support due to latency and bandwidth constraints, the firm says. As a consequence, some intelligence is migrating from the datacentre directly to the device. The firm’s view is that chipmakers are no longer just enabling connectivity; they are embedding advanced AI accelerators and NPUs into microcontrollers to power AI at the edge.
Yet revenue predictions for on-device AI in IoT remain relatively modest. Berg Insight has reported that over the past decade, the on-device AI market has been driven primarily by traditional machine learning use cases such as computer vision and anomaly detection, for which it has seen steady annual growth of around 10%. In recent years, the market has reached an inflection point as emerging technologies and applications in generative AI, robotics and autonomous driving have opened up new dimensions of growth. The firm estimates that the revenue generated by on-device AI solutions reached US$10.1 billion in 2024, an increase of around 22% from 2023.
This figure includes AI SoCs/SoMs, AI accelerators, AI MCUs and specialised on-device AI software and platforms, but excludes revenues generated by non-IoT applications such as smartphones, tablets and personal computers. The market is expected to grow to US$30.6 billion in 2029, representing a compound annual growth rate (CAGR) of 25%.
Figure 1: On-device AI revenue forecast
This wider adoption of on-device AI is enabling a new set of functions. Brenton sees preventative maintenance, fault detection and image or sensor inspection as the main use cases that will take advantage of the trend towards greater edge intelligence. “Anywhere bandwidth or battery life rules out constant cloud calls can benefit,” he confirms.
Into the intelligent last mile
A recent whitepaper from Transforma Insights, sponsored by Tata Communications, has found that IoT connectivity must evolve significantly to meet the new requirements. Next-generation solutions must support deep interoperability across heterogeneous device fleets, orchestrate distributed AI workloads between edge and cloud environments, and enable collaboration across the IoT stack and between connectivity providers. In addition, platforms must become more user-friendly through unified management, automation, and operational simplification. These capabilities collectively define the intelligent last mile required for AI-enabled IoT systems.
One of the key elements of the convergence of AI with IoT is the increasing deployment of AI onto the edge device itself, which Transforma and others call ‘AIoT’. Applying AI to IoT data on board the source IoT devices can bring significant benefits, including improved performance, enhanced compliance, privacy and security and potentially reduced operational costs. The whitepaper examines the motivations for deploying AI onto IoT devices, examples of propositions, the complexities of managing it in the field and the need to balance edge and cloud processing.
In quantitative terms, based on its AIoT forecasts, Transforma predicts that total AIoT connections will grow from 1.8 billion at the end of 2024 to 11.3 billion at the end of 2035, demonstrating a substantial uplift in AI capabilities embedded into IoT devices over the period.
Figure 2: AIoT connections forecast
Kochnowicz also sees advances in silicon being applied to IoT use cases and agrees that enterprises will select their approach based on the use case criteria. “Longevity is key for the industrial market where devices could be in the field for a decade or more,” he explains. “To be able to keep running the latest workloads, general purpose, programmable accelerators are a must. Cost, complexity and time-to-market are the ducks that need to be got in a row, to deploy this silicon intelligently.”
A clear compute hierarchy is emerging in which system-on-chip (SOC) solutions provide the lowest unit cost but rely on massive volume deployments for this to achieved. Next comes the SOM which pre-packs compute and memory into a customer carrier board. This results in increased cost but saves engineering time and a SOM is applicable to large but lower volume deployments that a SOC. Finally, the single board computer (SBC) is a fully-integrated, off-the-shelf platform that is ready to deploy. The drawback here is that an SBC can be expensive and can have limitations with regard to scalability.
“Even in the SoM and SBC, it is the SOC that brings the intelligence – if supported by an appropriate level of memory,” explains Kochnowicz. “The meaningful change coming to IoT therefore is through changes at the SoC level.”
By the numbers
The data points below are drawn from the IoT Analytics, Berg Insight and Transforma Insights forecasts cited in the market report.
0%
Less than 1% of the 21.1 billion IoT connections had a true edge AI component
~0 %
annual growth in the on-device AI market over the past decade
US$0bn
forecast on-device AI solution revenue in 2029, representing a CAGR of 25%
0bn
forecast total AIoT connections at the end of 2035, up from 1.8 billion in 2024
Swarm intelligence
“But of course, things never stand still,” he adds. “AI software-level innovations are already happening at pace, with significant work being done to make models small enough to run accurately on constrained chips, including things like low precision inference and self-compression algorithms. Next-generation IoT will shift to networks of fully decentralised, intelligent devices. We expect to see on-device continuous learning, reduced precision training and inference, chiplet-based design and distributed swarm intelligence that will see devices communicating directly with each other. The question is not if these will happen, but when.”
Transforma’s view is that the advent of more AI on IoT devices has a number of implications for supporting infrastructure. The firm says that among several impacts it reduces raw data streaming to the cloud, it necessitates frequent updating and lifecycle management, often across heterogeneous fleets, and it changes the nature of data storage architectures. Beyond these, AI increases the importance of security and governance issues and necessitates a change in approaches to traffic management as the workloads themselves transform. As a result there is an increasing requirement for new supporting infrastructure, most prominently in the form of AIoT platforms, featuring a range of new functionality.
Some of that functionality will be handled in the network with AI applied to optimise traffic flows and workloads, some of it will occur in the cloud where decision making will be augmented with AI to accelerate processes and aid affordability. However, networks and cloud compute resources still come at a significant cost and as IoT deployments scale up, those costs also increase. That’s putting the spotlight on what can be done on the device. The components involved in enabling added intelligence span processing, memory and power. Compellingly this bill of materials can now be purchased and integrated in a cost-effective timely manner.
* Video by MD BORHAN UDDIN from Pixabay
EDITORS TAKE
Edge intelligence moves closer to the device
In this edition of EDITORS TAKE with George Malim, we explore how on-device AI, smarter silicon and distributed intelligence are changing the economics of IoT — reducing cloud traffic and latency while enabling more autonomous, responsive devices.
Have your say on edge intelligence
Give us your opinion on the topic. Ring in and leave a message or use email to comment.