mmkw43
Posts: 768
Joined: Tue Dec 24, 2013 6:18 pm

Cython experiment

Mon May 17, 2021 8:29 pm

Well, I tried something in an effort to conceal code at least a little for my python script by experimenting with cython, thinking that maybe I can make part of the code a module (written in C?) that I can import. I put a question mark after that because although I have it working, I'm really not sure what the result is :) One of those things that I just tried, thinking trial and error but don't have a thorough understanding of the process and what I just did.

So I followed the process here at cython for the hello world example --
http://docs.cython.org/en/latest/src/tu ... orial.html

Part of my code checks for the cpu number to see if it's a match and shuts down if it doesn't --

Code: Select all

#!/usr/bin/env python3.7

import subprocess
import string

def marsf():
    sget = open('clkrun')    
    gox = sget.read()

    subprocess.call('sudo cat /proc/cpuinfo',shell=True)  

    sr = subprocess.check_output('sudo cat /proc/cpuinfo',shell=True)

    sr2 = sr.decode(encoding='UTF-8')

    agox = sr2.find(gox)

    if agox > 1:
        pass
    else:
        subprocess.call('sudo shutdown now',shell=True)

    print(gox)
    print(agox)
I put the print commands in there to just see if it was executing (and is).
I saved the file as mars.pyx and followed the instructions with the last part of the process being --
python3.7 setup.py build_ext --inplace (and it created the file).

So, in my script I commented out all that code above and did an import mars and in place of the code put in --

Code: Select all

mars.marsf()
and to my surprise, it worked first time and shows those print values in terminal.

So, do I actually now have a module written in C? I had read somewhere else that you need to set the variables with cdef first ? -- maybe not?

It created 2 folders as well. One that is 133 kb named mars.c and another 112kb named mars.cpython3.7m-arm-linux-gnue (or something like that). So, I guess what I'd like to know if in fact it is a C module and what those folders actually do ?

Teach me please. Thanks
Fact is, if this will offer me some better degree of protection, I'm going to add some more code that follows that cpu check. (In my "run" program function).

Heater
Posts: 18202
Joined: Tue Jul 17, 2012 3:02 pm

Re: Cython experiment

Mon May 17, 2021 10:05 pm

Seems to me I could create a "cat" program that returns whatever string I like.
By using "export PATH=/path/to/my/cat:$PATH" I could trick your code into running my "cat".
And so I can feed you code whatever string is required to get it to run for me.
Busted!

I don't understand why you want to run a subprocess to run "cat" to read "/proc/cpuinfo". Why not just read "/proc/cpuinfo" directly?
Memory in C++ is a leaky abstraction .

mmkw43
Posts: 768
Joined: Tue Dec 24, 2013 6:18 pm

Re: Cython experiment

Mon May 17, 2021 11:54 pm

I'm just trying to deter people who don't write code for a living. Maybe I should post the entire script so you can tell me what a novice I am.

I already know that -- perfection is not my goal.
I'll never be Joe Bonamassa on guitar either.

Actually, I'm rather proud of my desire to LEARN -- maybe not the code itself.

User avatar
jahboater
Posts: 7035
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: Cython experiment

Tue May 18, 2021 12:22 am

I think Heater was just pointing out the rather odd and very slow way of reading /proc/cpuinfo - so that you may improve it if you wish.

Code: Select all

    subprocess.call('sudo cat /proc/cpuinfo',shell=True)  

    sr = subprocess.check_output('sudo cat /proc/cpuinfo',shell=True)
You could just open the file /proc/cpuinfo which would be much much faster than spawning at least three separate processes (bash, sudo, and cat) to do the same thing.
I would add 1) that you don't need sudo, and 2) it looks like you are doing the 3 task thing twice (six processes!).
And, as Heater said, it doesn't work if you are trying to protect your code.

I am very pleased when someone finds problems in my code!
Saves me a lot of effort and they may spot problems I never would have - always a good thing.
That's one advantage of open source by the way.

Heater
Posts: 18202
Joined: Tue Jul 17, 2012 3:02 pm

Re: Cython experiment

Tue May 18, 2021 10:15 am

mmkw43 wrote:
Mon May 17, 2021 11:54 pm
I'm just trying to deter people who don't write code for a living. Maybe I should post the entire script so you can tell me what a novice I am.

I already know that -- perfection is not my goal.
I'll never be Joe Bonamassa on guitar either.
Please don't take my comments on your code as any kind of personal slight or criticism.

I saw some code posted here, with a request for comment, so I pointed out a few things that I feel are significant about it.
mmkw43 wrote:
Mon May 17, 2021 11:54 pm
Actually, I'm rather proud of my desire to LEARN -- maybe not the code itself.
We are all in the same boat. Nobody knows everything, no matter what experience they have over however many years. The computing/programming landscape changes all the time. We learn new things everyday. Often by discussing snippets of source code here.
Memory in C++ is a leaky abstraction .

mmkw43
Posts: 768
Joined: Tue Dec 24, 2013 6:18 pm

Re: Cython experiment

Tue May 18, 2021 2:14 pm

Well, I appreciate it but my latest endeavor is to understand what exactly is happening with cython.

It works but I'm in the dark about what the 2 files exactly are that are created and if in fact, I have truly created a C module (mars) that would be more difficult to analyze, compared to python. The file mars.c is unbelievable -- 133k of c code (I can view it), yet my original small bit of code was < 500 bytes. I was rather happy with myself it all worked on the first try.

I'm sure the cpu code can be written better and it's always been my intention afterI get this done to go back and post questions about how certain things could be improved.

There will be a limit considering the users of this how far they will go to look at my code (it wouldn't be worth it) and a few roadblocks will be all I need. I'm only concerned because the PI is so popular and many are familiar with some of the basics of terminal etc.

I did just a small snippet of code in my experiment with cython and my intention would be to do the entire "run" function which starts out with that cpu check but goes on to read settings and much else -- then starts the actual program loop.

The product has a countdown timer to enter setup or not, than a save settings function and then goes to the read/run function.

I first tried pyarmor (problems) -- pyminifier (a joke) -- but cython was highly suggested in some stuff I read about obfuscation techniques.

I also don't really know the proper location I should choose when creating the module. I used --inplace which puts it in /home/pi. Where are python modules located? I'm guessing there may be a better target directory but like I say, it works.

mmkw43
Posts: 768
Joined: Tue Dec 24, 2013 6:18 pm

Re: Cython experiment

Tue May 18, 2021 5:13 pm

Well, finally ran into a video that makes it more understandable --

https://www.youtube.com/watch?v=2OgGqkl ... Techlahoma

What concerns me is the size of the C file it creates (although it's working fine). I think it's because for cython to run more efficiently (and make a smaller C file?), I should name variables in my code for cython to use (cdef), rather than just feeding it my straight python script. (?) I think that's how I read it but maybe someone who has used cython may be able to help. -- thanks

edit --
I'm in over my head with this (even though it's working) but the size of that file scares me. Read this but I'm done. --

https://cython.readthedocs.io/en/latest ... onize.html

mmkw43
Posts: 768
Joined: Tue Dec 24, 2013 6:18 pm

Re: Cython experiment

Tue May 18, 2021 8:46 pm

also -- it always been my understanding you have to use subprocess to interact with terminal.

shows what I know.
cat /proc/cpuinfo is a command right?

Heater
Posts: 18202
Joined: Tue Jul 17, 2012 3:02 pm

Re: Cython experiment

Wed May 19, 2021 3:16 am

mmkw43 wrote:
Tue May 18, 2021 8:46 pm
cat /proc/cpuinfo is a command right?
Is indeed a command.

However "/proc/cpuinfo" is just a file that one can read with normal file access functions in whatever language on is using. For example in Python: https://www.pythontutorial.net/python-b ... text-file/

This saves having to run a subprocess or use the "cat" command.
Memory in C++ is a leaky abstraction .

User avatar
jahboater
Posts: 7035
Joined: Wed Feb 04, 2015 6:38 pm
Location: Wonderful West Dorset

Re: Cython experiment

Wed May 19, 2021 9:07 am

Heater wrote:
Wed May 19, 2021 3:16 am
mmkw43 wrote:
Tue May 18, 2021 8:46 pm
cat /proc/cpuinfo is a command right?
Is indeed a command.

However "/proc/cpuinfo" is just a file that one can read with normal file access functions in whatever language on is using. For example in Python: https://www.pythontutorial.net/python-b ... text-file/

This saves having to run a subprocess or use the "cat" command.
Just a bit further.
"cat" is the command.
In this usage, "cat" simply opens and reads the file "/proc/cpuinfo" and writes the output to the "standard output" which you are retrieving with popen().

The sudo is not needed because the file /proc/cpuinfo is readable by all, with no special permissions needed.

mmkw43
Posts: 768
Joined: Tue Dec 24, 2013 6:18 pm

Re: Cython experiment

Wed May 19, 2021 2:44 pm

Yeah, I got into this bad habit early of putting sudo where it didn't belong. sometimes it solved a problem, other times not but just left it there (NOT a good idea I realize). I get sudo now but didn't for a long time actually.

And I see that subprocess is overkill for some of the things I'm doing. Again, I just had this mindset as a beginner to go with what I knew "functioned" (NOT a good idea). Didn't take the time to see that I had the option of shutil etc for copy and had in my mind that "cat" was some sort of special command I needed to for the cpuinfo -- when alas, it's just a friggin' file.

I still want to "try" and obfuscate at least one part of my script with cython. (Maybe).

I see now that so much C code was written perhaps because my short script is also importing a couple of modules and maybe it's putting those in C also? And I "think" you can reduce the size of the C file by "setting up" the python code first (defining cdef variables etc?). Not sure though.

Return to “Python”