“Transmitting bits of information, even with approaches like bluetooth low energy, is in the tens to hundreds of milliwatts in the best of circumstances at comparatively short range. The efficiency of radio transmission doesn’t seem to be improving dramatically over time either, there seem to be some tough hurdles imposed by physics that make improvements hard.

On a happier note, capturing data through sensors doesn’t suffer from the same problem. There are microphones, accelerometers, and even image sensors that operate well below a milliwatt, even down to tens of microwatts. The same is true for arithmetic. Microprocessors and DSPs are able to process tens or hundreds of millions of calculations for under a milliwatt, even with existing technologies, and much more efficient low-energy accelerators are on the horizon.”

What this means is that most data that’s being captured by sensors in the embedded world is just being discarded, without being analyzed at all.

Scaling machine learning models to embedded devices

UPDATE an example of things moving in that space, startup with chip to move AI to the “edge” https://techcrunch.com/2019/05/20/quadric-io-raises-15m-to-build-a-plug-and-play-supercomputer-for-autonomous-systems/

Microfluidic devices https://www.nature.com/protocolexchange/system/uploads/3745/original/Microfluidic_device_design_fabrication_and_testing_protocols_protex2015_069.pdf?1436957242

Manifolds, topology and AI Colah Blog http://colah.github.io/posts/2014-03-NN-Manifolds-Topology/

cellular automata

the zoo of neural networks http://www.asimovinstitute.org/neural-network-zoo/

“It is really Python + intuition” me pitching this book to L.

https://www.amazon.com/Deep-Learning-Python-Francois-Chollet/dp/1617294438

really, AI people are fixated with the intuition behind neural networks and all the architectural variations that comes with it. I love all the graphocal explanations of techniques that really get me to visualize and understand what happens. But then, can we make Ai explainable tru intuintion?

the Concept of Intuition in Artificial Intelligence paper

Fake Intuitive Explanations in AI

https://www.lrb.co.uk/v37/n05/john-lanchester/the-robots-are-coming

“A thorough, considered and disconcerting study of that possibility was undertaken by two Oxford economists, Carl Benedikt Frey and Michael Osborne, in a paper from 2013 called ‘The Future of Employment: How Susceptible Are Jobs to Computerisation?’​”

review of

-Tyler Cowen The End of Average

-Second Machine Age Bjodhebfhwe etc etc

productivity, employment and robot deflation are dealt with “It says a lot about the current moment that as we stand facing a future which might resemble either a hyper-capitalist dystopia or a socialist paradise, the second option doesn’t get a mention”

the Bitter Lesson of Rich Sutton says that progress in 70 years of Ai has come from general methods relying on computation power which has increased from Moore Law. Any method based on developing specific knowledge of the field has failed, while “brute force”

the Better Lesson of Rodney Brooks retorts that Moore Law might yield doubling every 20 years, Dennard scaling (energy invariance) broke down in 2006 so there is not so much computing power to waste and reserach is very active on network architectures and custom-designed chips.

In a way both are right, speaking of different levels of generalizations. Sutton:

“In computer vision, there has been a similar pattern. Early methods conceived of vision as searching for edges, or generalized cylinders, or in terms of SIFT features. But today all this is discarded. Modern deep-learning neural networks use only the notions of convolution and certain kinds of invariances, and perform much better.