wig.netlify.com

I am using pyserial on RPi to read data from sensor. The code basically looks like this:

However, it comsumes almost 100% of the cpu. I also run the program on a Windows laptop which is more powerful, the cpu usage is about 12% and the reading speed on RPi is slower than the speed on my laptop.

12% sounds like one full CPU on a quad core with 8 'threads' (hyperthreading)

• you say the loop basically looks like this? what else is there? is the CPU load really from this thread or some other thread in the application?
• Which version of Python and pySerial are involved?
• have you tried read instead of readline?
• see also http://pyserial.readthedocs.io/en/latest/pyserial_api.html#serial.threaded.LineReader (usage example at the bottom of the page)
• try http://pyserial.readthedocs.io/en/latest/url_handlers.html#spy to see what its doing on the serial port

• I created a read thread to do all the reading.

• I am using the latest python 3.6.1 and the latest pyserial installed by pip3 in a vitualenv.

  

• Yes, I have tried read and detected newline characters by myself but it didn't solve the problem.

I will try the two links tomorrow and provide feedback.

Thank you for your help.

I checked my laptop's CPU, it is i7-6500U which has 2 core and 4 threads for each core, so the program also consume all the resource of a thread on my laptop.
I run the following script on both my laptop and RPi:

I got the following result:
test.txt
test-pi.txt
Both of them get a same amount of readings except that RPi took a little bit more time to fetch readings from the buffer.

For the serial.threaded.LineReader, I got error AttributeError: module 'serial' has no attribute 'threaded'.

I wrote another script to test and I found something interesting:

I changed sleep time from 0.25s to 10s, it always take about 2s to read all data from the buffer even the size of data varied, on both my laptop and RPi.
I used cProfile to take a look at the method call time and found this:
1 0.000 0.000 2.000 2.000 {method 'readall' of '_io._RawIOBase' objects}

I fixed the problem by get the number of bytes to read first and then read it from buffer:

Do you need me to close the issue?
Thank you for your library and help!

maybe the timeout process let's overload the cpu.

@leomikezee I think the problem is because the pyserial is working with polling mode. It doesn't like java or c with event trigger mechanism.

It is easy to implement a non-CPU time wasting readline method. In a loop, first call ser.read(1) and then call ser.read(ser.in_waiting). Each time, check whether the returned data contains n. Of course the data after n needs to be saved as it may be part of another line. Also see the my recv() function from issue #318

Not only does ser.readline() waste 100% of one CPU core. It is also slow as hell! Reading from a virtual COM port /dev/ttyACM1 attached via USB, ser.readline() would give me a throughput of about 150 kB/sec. Using ser.read(1) in combination with ser.read(ser.in_waiting) would give me a throughput of about 660kB/sec.

Here's a class that serves as a wrapper to a pyserial object. It allows you to read lines without 100% CPU. It does not contain any timeout logic. If a timeout occurs, self.s.read(i) returns an empty string and you might want to throw an exception to indicate the timeout.

I just measured. The code below gives me 790 kB/sec while replacing the code with pyserial's readline method gives me just 170kB/sec.

@skoehler That code looks really useful! I wonder if it should be included in the examples for pyserial, if it's the best way to access the serial port via the library? I noticed that the pyserial examples haven't been updated in years. Do you have a complete working example of the class that you could shaare?

this seems to be an issue with the lib; I took the complete system image and put on another raspberry (same model), and in the second device the CPU usage of the following part of the code is 50%, against >20% for the whole application in the other raspberry. The input is the same.