Home    Bloggers    Messages    Resources
Tw  |  Fb  |  In  |  Rss
Rich Quinnell

Internet of Things Could Use Wakeup Benchmark

Rich Quinnell
Newest First   Oldest First   Threaded View
Rich Quinnell
Rich Quinnell
5/17/2012 7:02:55 PM
User Rank
Blogger
Re: The Benchmark is coming
Thanks for the update, EEMBCdude. Hope the comments here were useful. At the very least they should give some idea of what developers are thinking and their biases and opinions so you can tailor your messaging to them when you get ready to announce.

50%
50%
EEMBCdude
EEMBCdude
5/17/2012 10:01:45 AM
User Rank
Bit twiddler
The Benchmark is coming
Good article and good feedback. I can't disclose too much since it's still internal info, but EEMBC is working on the benchmark to address the MCU power efficiency topic, it will also include measurement of peripheral overhead and power contributions. Suffice it to say that it's tough to devise a standard that the industry will agree on - we've been working on it for well over a year, but are getting close.

100%
0%
prabhakar_deosthali
prabhakar_deosthali
4/21/2012 5:36:29 AM
User Rank
Program Manager
Re: Why benchmark
Thanks Rich for ellaborating on the importance of this new proposed benchmark. This benchmark will not only measure the hardware performance of the MCU but how quickly the Software( OS and the related drivers) is able to establish the connection with the outside world and complete the message transaction

50%
50%
Rich Quinnell
Rich Quinnell
4/20/2012 4:08:49 PM
User Rank
Blogger
OS effects
RD, I agree. Measuring the impact of the OS on the result could be a big benefit. Probably all you would have to do is run the test under each OS using the same hardware to see what happens. It would depend on the benchmark's form, though. If it were a compiled program executing as a single routine (as I would expect for MCU evaluation) the OS might not make much difference.

I totally agree about the differing message lengths being needed. An MCU with a slow startup but fast execution might lag on short messages but excel with long ones. Having different message lengths available in the evaluation process would allow a developer to select the case most appropriate to his application.

50%
50%
Rich Quinnell
Rich Quinnell
4/20/2012 3:29:21 PM
User Rank
Blogger
Why benchmark
Prabhakar, you asked why a separate benchmark given that the parameter we are measuring is so dependent on other functions. Short answer: so we can compare alternatives and see which is best for our application. The benchmark gives a standard way of measuring a parameter under standard conditions so that we can make a direct comparison between alternatives. The benchmark may well show that MCU-A gets a better score using 10M Ethernet while MCU-B is superior for the 1G. That's not a problem with the benchmark. In fact, it might be valuable information for the selection process.

The point of a benchmark is to freeze or nromalize all variables (including the algorithm) except the one thing you want to evaulate - the MCU itself. What value you choose for those variables is part of the evaluation. You want them to be representative of your actual application.

As to using the power draw while fully active, that is not sufficient to answer the question. The proposed benchmark would involve the entire activity of waking, responding, and going back to sleep. Simply considering the active mode ignores the impact of the beginning and ending activity. And the contention is that those activities have a significant impact that is not currently being quantified. The benchmark would address that information void.

50%
50%
Robotics Developer
Robotics Developer
4/20/2012 2:54:53 PM
User Rank
Program Manager
Re: Quibbles
Rich, I think that you are on the right track.  I would also suggest that the benchmark may have differing levels for the testing.  Remote simple sensors only need to send small data packets while wireless phones may have to deal with significant amounts of data.  One thing that occurred to me while reading the article was measuring the effect of the Operating System (or lack) in the benchmark.  There might be significant differences in power between say eLinux and WindowsCE.  I would think that would be both interesting and very helpful for designers.

50%
50%
Microp
Microp
4/20/2012 3:04:43 AM
User Rank
Program Manager
Benchmarking in all respect
Rich, benchmarking can help to standardize the product. I think along with energy consumption, we have to use benchmarking for other functionalities like maximum data throughput at different user levels, speed, connectivity, traffic flow etc. Actually at benchmarking process (in general) we used to keep a track of all these factors.

100%
0%
prabhakar_deosthali
prabhakar_deosthali
4/20/2012 2:43:34 AM
User Rank
Program Manager
re:
The time taken by the MCU to wake up , complete a message transmission and sleep will also be a function of network bandwidth available at that point in time and the network quality.

So why do we need this separate benchmark?

Can "The power consumption per unit of time in a fully active mode"  suffice?

50%
50%
Rich Quinnell
Rich Quinnell
4/19/2012 3:51:20 PM
User Rank
Blogger
Re: Quibbles
Curt, you bring up some good points, but I think they can be readily addressed. As to the linear model, I suspect that this graph is just for purposes of illustration that there is a time between sleep and full functional operation where the MCU is using power but not performing a task other than getting ready to perform a task. Whether it is a linear, nonlinear, or step function is really not relevant, so the figure shows the simplest form possible to make clear the issue of "wasted power."

As to the energy of creating the message and such, that should all be addressed in the benchmark's design. The idea of the benchmark is to make a direct comparison between two devices by running the same task the same way on each. How that task functions is part of the benchmark design process. The design needs to be representative of a real-world behavior, for instance, so it should include such things as waking up, establishing the wireless link, formulating the message, sending it, getting whatever acknowledgement there may be, and shutting down. There may be tricks a programmer can pull to optimize the task, but those should already be in the benchmark software. Or perhaps the benchmark can be used to validate and quantify the improvements such tricks produce.

Similarly, such things as message rate and message power would be standardized in the benchmark to make comparisons viable. We know lowering RF output power will help. We know lowering clock speed will help. What we don't know is how much work the MPU has to do to handle the task. Fix the parameters that aren't MCU specific and we can get at the ones that are.

50%
50%
Curt Carpenter
Curt Carpenter
4/19/2012 11:24:58 AM
User Rank
Blogger
Quibbles
Well, I think we should start an immediate whine about the linear power ramp up model there, and then quibble about the energy that the MCU has to invest in creating any message it needs to send in the first place, and maybe argue about whether the MCU forms and stores its NEXT message before it takes that step-function drop back into sleep mode, and whether the MCU peak power is with ALL peripherals on, or none, or just some as required.  And then, shouldn't it really take into account the rate at which messages need to be sent, whatever the power per message is?  ....  and so on.

The "Trouble with Tribbles" thus morphs into the "Question of Quibbles" for all benchmark efforts.  

Otherwise, sounds like a useful benchmark to me :-)

 

50%
50%
More Blogs from Rich Quinnell
We need to talk about setting up discussion groups on the site, so I've set up a live chat for Friday.
If you think MCC needs some traditional discussion groups, come help set them up.
Hot on the heels of the Beaglebone Black has come a book telling how to use it. A pretty good book, too.
Rich's monthly summary of MCU-related product releases includes some of the Design West announcements he was unable to cover earlier.
With its recent acquisition of Code Red, NXP aims to improve developer's ability to use its MCUs right out of the box.
flash poll
MC on twitter
like us on facebook
Microcontroller Central    About Us     Contact Us     Help     Register     Twitter     Facebook     RSS