pumpkinpi
Posts: 141
Joined: Fri Apr 18, 2014 7:11 pm

Sharing data between python scripts through a LAN

Thu Aug 31, 2017 12:34 am

Hi Python Masters,

I have a bunch of Pi's running various python scripts in my house. They do a whole bunch of fun house monitoring jobs.

For quite a while, I was passing data between them via a simple NFS server and just writing the information to text files. I have had issues with the number of writes in those drives (USB thumb drives) and want to elevate my code a bit and have the scripts talk to themselves directly.

I think what I want to do is use socketserver:
https://docs.python.org/3/library/socketserver.html

to allow the simple passage of information from one Pi to the next and back.

Looking at this page, there are a whole lot of other options that may be too involved:
https://stackoverflow.com/questions/936 ... -in-python

I am not worried about security for my computers as they are all on a LAN and not communicating to the wider world (ie. behind a firewall).

Just looking for direction on what would be the simplest to implement and master.

Any advice appreciated.

PPi

asandford
Posts: 1997
Joined: Mon Dec 31, 2012 12:54 pm
Location: Waterlooville

Re: Sharing data between python scripts through a LAN

Thu Aug 31, 2017 12:50 am

Use MQTT. The python programs that currently write data would publish that to topics and the programs reading data will subscribe to those topics to receive the latest info.

Here's a tutorial that explains message queues, example 3 is relevant (the RabbitMQ package is conceptually the same as MQTT, but has more funtionality).

User avatar
MrYsLab
Posts: 376
Joined: Mon Dec 15, 2014 7:14 pm
Location: Noo Joysey, USA

Re: Sharing data between python scripts through a LAN

Thu Aug 31, 2017 1:43 am

Take a look at python banyan https://github.com/MrYsLab/python_banyan. It is simple to use and has much better performance than MQTT.

pumpkinpi
Posts: 141
Joined: Fri Apr 18, 2014 7:11 pm

Re: Sharing data between python scripts through a LAN

Thu Aug 31, 2017 2:29 am

Both look like they would work. Doing some reading. Thanks for the help so far.

User avatar
DougieLawson
Posts: 36322
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: Sharing data between python scripts through a LAN

Thu Aug 31, 2017 10:44 am

asandford wrote:
Thu Aug 31, 2017 12:50 am
(the RabbitMQ package is conceptually the same as MQTT Mosquitto, but has more funtionality).
MQTT is the protocol.

RabbitMQ is one broker.
Mosquitto is another broker.

RabbitMQ does more than Mosquitto.
Note: Having anything humorous in your signature is completely banned on this forum. Wear a tin-foil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

asandford
Posts: 1997
Joined: Mon Dec 31, 2012 12:54 pm
Location: Waterlooville

Re: Sharing data between python scripts through a LAN

Thu Aug 31, 2017 9:09 pm

pumpkinpi wrote:
Thu Aug 31, 2017 2:29 am
Both look like they would work. Doing some reading. Thanks for the help so far.
If you want to have a play around, an MQTT client is supplied with the in-built Node-Red, all you need is a broker.

asandford
Posts: 1997
Joined: Mon Dec 31, 2012 12:54 pm
Location: Waterlooville

Re: Sharing data between python scripts through a LAN

Thu Aug 31, 2017 9:15 pm

MrYsLab wrote:
Thu Aug 31, 2017 1:43 am
Take a look at python banyan https://github.com/MrYsLab/python_banyan. It is simple to use and has much better performance than MQTT.
And only works with python 3.4 (and above) :lol:

User avatar
MrYsLab
Posts: 376
Joined: Mon Dec 15, 2014 7:14 pm
Location: Noo Joysey, USA

Re: Sharing data between python scripts through a LAN

Fri Sep 01, 2017 12:37 pm

Python banyan is now python2 and 3 compatible :D :D :D . It also allows for numpy matrix transfers as a single payload. And streaming messages on a single Rpi 3 is twice as fast as Mosquitto MQTT (test code is available in github).

pumpkinpi
Posts: 141
Joined: Fri Apr 18, 2014 7:11 pm

Re: Sharing data between python scripts through a LAN

Fri Sep 01, 2017 11:53 pm

I am going to give banyan a try. Problem is I get a ton of compilation errors when I run:

Code: Select all

sudo pip install python-banyan
A couple of the errors that flew past my screen are:

Code: Select all

    arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DZMQ_HAVE_CURVE=1 -DZMQ_USE_TWEETNACL=1 -DZMQ_USE_EPOLL=1 -Ibundled/zeromq/include -Ibundled/zeromq/tweetnacl/src -Ibundled/zeromq/tweetnacl/contrib/randombytes -Ibundled -I/usr/include/python2.7 -c buildutils/initlibzmq.c -o build/temp.linux-armv6l-2.7/buildutils/initlibzmq.o
    buildutils/initlibzmq.c:10:20: fatal error: Python.h: No such file or directory
     #include "Python.h"
                        ^
    compilation terminated.
    error: command 'arm-linux-gnueabihf-gcc' failed with exit status 1
    Complete output from command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-xK0ONM/pyzmq/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-fmwbE0-record/install-record.txt --single-version-externally-managed --compile:
    running install
And:

Code: Select all

buildutils/initlibzmq.c:10:20: fatal error: Python.h: No such file or directory

 #include "Python.h"

                    ^

compilation terminated.

error: command 'arm-linux-gnueabihf-gcc' failed with exit status 1

----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-xK0ONM/pyzmq/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-fmwbE0-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip-build-xK0ONM/pyzmq
Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    load_entry_point('pip==1.5.6', 'console_scripts', 'pip')()
  File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 248, in main
    return command.main(cmd_args)
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 161, in main
    text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 68: ordinal not in range(128)
Not sure what I am missing.

PPi

User avatar
MrYsLab
Posts: 376
Joined: Mon Dec 15, 2014 7:14 pm
Location: Noo Joysey, USA

Re: Sharing data between python scripts through a LAN

Sat Sep 02, 2017 12:24 am

I just loaded this on the stretch distribution and here is my output:

Code: Select all

[email protected]:~ $ sudo pip install python-banyan
Collecting python-banyan
  Using cached python-banyan-2.0.tar.gz
Collecting msgpack-numpy (from python-banyan)
  Using cached msgpack_numpy-0.4.1-py2.py3-none-any.whl
Collecting msgpack-python (from python-banyan)
  Using cached msgpack-python-0.4.8.tar.gz
Requirement already satisfied: numpy in /usr/lib/python2.7/dist-packages (from python-banyan)
Requirement already satisfied: psutil in /usr/local/lib/python2.7/dist-packages (from python-banyan)
Collecting pyzmq (from python-banyan)
  Using cached pyzmq-16.0.2.tar.gz
Collecting u-msgpack-python (from python-banyan)
  Using cached u_msgpack_python-2.4.1-py2.py3-none-any.whl
Building wheels for collected packages: python-banyan, msgpack-python, pyzmq
  Running setup.py bdist_wheel for python-banyan ... done
  Stored in directory: /root/.cache/pip/wheels/03/af/57/67a7478bf20c79a1b427c378a579c29fe879fd03c01b821397
  Running setup.py bdist_wheel for msgpack-python ... done
  Stored in directory: /root/.cache/pip/wheels/2c/e7/e7/9031652a69d594665c5ca25e41d0fb3faa024e730b590e4402
  Running setup.py bdist_wheel for pyzmq ... done
  Stored in directory: /root/.cache/pip/wheels/c8/68/e5/a747dfb649857b6e4f954ef922f41ac50d12142aefa58656a6
Successfully built python-banyan msgpack-python pyzmq
Installing collected packages: msgpack-python, msgpack-numpy, pyzmq, u-msgpack-python, python-banyan
Successfully installed msgpack-numpy-0.4.1 msgpack-python-0.4.8 python-banyan-2.0 pyzmq-16.0.2 u-msgpack-python-2.4.1
[email protected]:~ $ 
All packages loaded without error.

Which version of Raspbian are you using?

It looks like things are bombing in the load of pyzmq. Could you try typing:

Code: Select all

sudo pip install pyzmq
and let me know if that succeeds.

pumpkinpi
Posts: 141
Joined: Fri Apr 18, 2014 7:11 pm

Re: Sharing data between python scripts through a LAN

Sat Sep 02, 2017 12:29 am

Running a fully updated version of Raspbian Jessie.

Tried this:

Code: Select all

sudo pip install pyzmq
Get this (buffer does not hold everything):

Code: Select all

    timer_createXLUUXL.c:(.text+0x14): undefined reference to `timer_create'
    collect2: error: ld returned 1 exit status
    no timer_create, linking librt
    ************************************************
    building 'zmq.libzmq' extension
    arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DZMQ_HAVE_CURVE=1 -DZMQ_USE_TWEETNACL=1 -DZMQ_USE_EPOLL=1 -Ibundled/zeromq/include -Ibundled/zeromq/tweetnacl/src -Ibundled/zeromq/tweetnacl/contrib/randombytes -Ibundled -I/usr/include/python2.7 -c buildutils/initlibzmq.c -o build/temp.linux-armv6l-2.7/buildutils/initlibzmq.o
    buildutils/initlibzmq.c:10:20: fatal error: Python.h: No such file or directory
     #include "Python.h"
                        ^
    compilation terminated.
    error: command 'arm-linux-gnueabihf-gcc' failed with exit status 1
    Complete output from command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-vfKpE0/pyzmq/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-wwXCX8-record/install-record.txt --single-version-externally-managed --compile:
    running install

running build

running build_py

creating build

creating build/lib.linux-armv6l-2.7

creating build/lib.linux-armv6l-2.7/zmq

copying zmq/error.py -> build/lib.linux-armv6l-2.7/zmq

copying zmq/decorators.py -> build/lib.linux-armv6l-2.7/zmq

copying zmq/__init__.py -> build/lib.linux-armv6l-2.7/zmq

creating build/lib.linux-armv6l-2.7/zmq/auth

copying zmq/auth/certs.py -> build/lib.linux-armv6l-2.7/zmq/auth

copying zmq/auth/thread.py -> build/lib.linux-armv6l-2.7/zmq/auth

copying zmq/auth/__init__.py -> build/lib.linux-armv6l-2.7/zmq/auth

copying zmq/auth/ioloop.py -> build/lib.linux-armv6l-2.7/zmq/auth

copying zmq/auth/base.py -> build/lib.linux-armv6l-2.7/zmq/auth

creating build/lib.linux-armv6l-2.7/zmq/devices

copying zmq/devices/monitoredqueue.py -> build/lib.linux-armv6l-2.7/zmq/devices

copying zmq/devices/__init__.py -> build/lib.linux-armv6l-2.7/zmq/devices

copying zmq/devices/monitoredqueuedevice.py -> build/lib.linux-armv6l-2.7/zmq/devices

copying zmq/devices/basedevice.py -> build/lib.linux-armv6l-2.7/zmq/devices

copying zmq/devices/proxydevice.py -> build/lib.linux-armv6l-2.7/zmq/devices

creating build/lib.linux-armv6l-2.7/zmq/log

copying zmq/log/__init__.py -> build/lib.linux-armv6l-2.7/zmq/log

copying zmq/log/handlers.py -> build/lib.linux-armv6l-2.7/zmq/log

creating build/lib.linux-armv6l-2.7/zmq/green

copying zmq/green/poll.py -> build/lib.linux-armv6l-2.7/zmq/green

copying zmq/green/device.py -> build/lib.linux-armv6l-2.7/zmq/green

copying zmq/green/core.py -> build/lib.linux-armv6l-2.7/zmq/green

copying zmq/green/__init__.py -> build/lib.linux-armv6l-2.7/zmq/green

creating build/lib.linux-armv6l-2.7/zmq/green/eventloop

copying zmq/green/eventloop/__init__.py -> build/lib.linux-armv6l-2.7/zmq/green/eventloop

copying zmq/green/eventloop/ioloop.py -> build/lib.linux-armv6l-2.7/zmq/green/eventloop

copying zmq/green/eventloop/zmqstream.py -> build/lib.linux-armv6l-2.7/zmq/green/eventloop

creating build/lib.linux-armv6l-2.7/zmq/backend

copying zmq/backend/__init__.py -> build/lib.linux-armv6l-2.7/zmq/backend

copying zmq/backend/select.py -> build/lib.linux-armv6l-2.7/zmq/backend

creating build/lib.linux-armv6l-2.7/zmq/backend/cython

copying zmq/backend/cython/__init__.py -> build/lib.linux-armv6l-2.7/zmq/backend/cython

creating build/lib.linux-armv6l-2.7/zmq/backend/cffi

copying zmq/backend/cffi/utils.py -> build/lib.linux-armv6l-2.7/zmq/backend/cffi

copying zmq/backend/cffi/_poll.py -> build/lib.linux-armv6l-2.7/zmq/backend/cffi

copying zmq/backend/cffi/error.py -> build/lib.linux-armv6l-2.7/zmq/backend/cffi

copying zmq/backend/cffi/socket.py -> build/lib.linux-armv6l-2.7/zmq/backend/cffi

copying zmq/backend/cffi/__init__.py -> build/lib.linux-armv6l-2.7/zmq/backend/cffi

copying zmq/backend/cffi/constants.py -> build/lib.linux-armv6l-2.7/zmq/backend/cffi

copying zmq/backend/cffi/message.py -> build/lib.linux-armv6l-2.7/zmq/backend/cffi

copying zmq/backend/cffi/_cffi.py -> build/lib.linux-armv6l-2.7/zmq/backend/cffi

copying zmq/backend/cffi/context.py -> build/lib.linux-armv6l-2.7/zmq/backend/cffi

copying zmq/backend/cffi/devices.py -> build/lib.linux-armv6l-2.7/zmq/backend/cffi

creating build/lib.linux-armv6l-2.7/zmq/sugar

copying zmq/sugar/version.py -> build/lib.linux-armv6l-2.7/zmq/sugar

copying zmq/sugar/frame.py -> build/lib.linux-armv6l-2.7/zmq/sugar

copying zmq/sugar/poll.py -> build/lib.linux-armv6l-2.7/zmq/sugar

copying zmq/sugar/socket.py -> build/lib.linux-armv6l-2.7/zmq/sugar

copying zmq/sugar/attrsettr.py -> build/lib.linux-armv6l-2.7/zmq/sugar

copying zmq/sugar/__init__.py -> build/lib.linux-armv6l-2.7/zmq/sugar

copying zmq/sugar/constants.py -> build/lib.linux-armv6l-2.7/zmq/sugar

copying zmq/sugar/tracker.py -> build/lib.linux-armv6l-2.7/zmq/sugar

copying zmq/sugar/context.py -> build/lib.linux-armv6l-2.7/zmq/sugar

copying zmq/sugar/stopwatch.py -> build/lib.linux-armv6l-2.7/zmq/sugar

creating build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_win32_shim.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_imports.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_error.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_auth.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_ssh.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_pair.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_ioloop.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_version.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_retry_eintr.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_device.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_z85.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_monitor.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_constants.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_context.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/__init__.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_poll.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_socket.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_etc.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_decorators.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_multipart.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_reqrep.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_message.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_log.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_zmqstream.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_pubsub.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_cffi_backend.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_monqueue.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_includes.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_security.py -> build/lib.linux-armv6l-2.7/zmq/tests

copying zmq/tests/test_future.py -> build/lib.linux-armv6l-2.7/zmq/tests

creating build/lib.linux-armv6l-2.7/zmq/eventloop

copying zmq/eventloop/future.py -> build/lib.linux-armv6l-2.7/zmq/eventloop

copying zmq/eventloop/__init__.py -> build/lib.linux-armv6l-2.7/zmq/eventloop

copying zmq/eventloop/ioloop.py -> build/lib.linux-armv6l-2.7/zmq/eventloop

copying zmq/eventloop/zmqstream.py -> build/lib.linux-armv6l-2.7/zmq/eventloop

creating build/lib.linux-armv6l-2.7/zmq/eventloop/minitornado

copying zmq/eventloop/minitornado/util.py -> build/lib.linux-armv6l-2.7/zmq/eventloop/minitornado

copying zmq/eventloop/minitornado/__init__.py -> build/lib.linux-armv6l-2.7/zmq/eventloop/minitornado

copying zmq/eventloop/minitornado/log.py -> build/lib.linux-armv6l-2.7/zmq/eventloop/minitornado

copying zmq/eventloop/minitornado/concurrent.py -> build/lib.linux-armv6l-2.7/zmq/eventloop/minitornado

copying zmq/eventloop/minitornado/ioloop.py -> build/lib.linux-armv6l-2.7/zmq/eventloop/minitornado

copying zmq/eventloop/minitornado/stack_context.py -> build/lib.linux-armv6l-2.7/zmq/eventloop/minitornado

creating build/lib.linux-armv6l-2.7/zmq/eventloop/minitornado/platform

copying zmq/eventloop/minitornado/platform/posix.py -> build/lib.linux-armv6l-2.7/zmq/eventloop/minitornado/platform

copying zmq/eventloop/minitornado/platform/auto.py -> build/lib.linux-armv6l-2.7/zmq/eventloop/minitornado/platform

copying zmq/eventloop/minitornado/platform/__init__.py -> build/lib.linux-armv6l-2.7/zmq/eventloop/minitornado/platform

copying zmq/eventloop/minitornado/platform/interface.py -> build/lib.linux-armv6l-2.7/zmq/eventloop/minitornado/platform

copying zmq/eventloop/minitornado/platform/windows.py -> build/lib.linux-armv6l-2.7/zmq/eventloop/minitornado/platform

copying zmq/eventloop/minitornado/platform/common.py -> build/lib.linux-armv6l-2.7/zmq/eventloop/minitornado/platform

creating build/lib.linux-armv6l-2.7/zmq/ssh

copying zmq/ssh/__init__.py -> build/lib.linux-armv6l-2.7/zmq/ssh

copying zmq/ssh/tunnel.py -> build/lib.linux-armv6l-2.7/zmq/ssh

copying zmq/ssh/forward.py -> build/lib.linux-armv6l-2.7/zmq/ssh

creating build/lib.linux-armv6l-2.7/zmq/utils

copying zmq/utils/garbage.py -> build/lib.linux-armv6l-2.7/zmq/utils

copying zmq/utils/sixcerpt.py -> build/lib.linux-armv6l-2.7/zmq/utils

copying zmq/utils/strtypes.py -> build/lib.linux-armv6l-2.7/zmq/utils

copying zmq/utils/__init__.py -> build/lib.linux-armv6l-2.7/zmq/utils

copying zmq/utils/interop.py -> build/lib.linux-armv6l-2.7/zmq/utils

copying zmq/utils/jsonapi.py -> build/lib.linux-armv6l-2.7/zmq/utils

copying zmq/utils/constant_names.py -> build/lib.linux-armv6l-2.7/zmq/utils

copying zmq/utils/z85.py -> build/lib.linux-armv6l-2.7/zmq/utils

copying zmq/utils/monitor.py -> build/lib.linux-armv6l-2.7/zmq/utils

copying zmq/utils/win32.py -> build/lib.linux-armv6l-2.7/zmq/utils

copying zmq/devices/monitoredqueue.pxd -> build/lib.linux-armv6l-2.7/zmq/devices

copying zmq/backend/cython/checkrc.pxd -> build/lib.linux-armv6l-2.7/zmq/backend/cython

copying zmq/backend/cython/libzmq.pxd -> build/lib.linux-armv6l-2.7/zmq/backend/cython

copying zmq/backend/cython/message.pxd -> build/lib.linux-armv6l-2.7/zmq/backend/cython

copying zmq/backend/cython/context.pxd -> build/lib.linux-armv6l-2.7/zmq/backend/cython

copying zmq/backend/cython/socket.pxd -> build/lib.linux-armv6l-2.7/zmq/backend/cython

copying zmq/backend/cython/constants.pxi -> build/lib.linux-armv6l-2.7/zmq/backend/cython

copying zmq/backend/cython/constant_enums.pxi -> build/lib.linux-armv6l-2.7/zmq/backend/cython

copying zmq/backend/cffi/_cdefs.h -> build/lib.linux-armv6l-2.7/zmq/backend/cffi

copying zmq/backend/cffi/_verify.c -> build/lib.linux-armv6l-2.7/zmq/backend/cffi

copying zmq/utils/buffers.pxd -> build/lib.linux-armv6l-2.7/zmq/utils

copying zmq/utils/pyversion_compat.h -> build/lib.linux-armv6l-2.7/zmq/utils

copying zmq/utils/zmq_compat.h -> build/lib.linux-armv6l-2.7/zmq/utils

copying zmq/utils/getpid_compat.h -> build/lib.linux-armv6l-2.7/zmq/utils

copying zmq/utils/zmq_constants.h -> build/lib.linux-armv6l-2.7/zmq/utils

copying zmq/utils/ipcmaxlen.h -> build/lib.linux-armv6l-2.7/zmq/utils

running build_ext

running configure

Did not find libzmq via pkg-config.

{'libraries': ['zmq'], 'extra_link_args': [], 'runtime_library_dirs': [], 'library_dirs': [], 'include_dirs': []}

arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -c build/temp.linux-armv6l-2.7/scratch/check_sys_un.c -o build/temp.linux-armv6l-2.7/scratch/check_sys_un.o

build/temp.linux-armv6l-2.7/scratch/check_sys_un.c: In function ‘main’:

build/temp.linux-armv6l-2.7/scratch/check_sys_un.c:6:5: warning: format ‘%lu’ expects argument of type ‘long unsigned int’, but argument 2 has type ‘unsigned int’ [-Wformat=]

     printf("%lu\n", sizeof(dummy->sun_path) - 1);

     ^

arm-linux-gnueabihf-gcc -pthread build/temp.linux-armv6l-2.7/scratch/check_sys_un.o -o build/temp.linux-armv6l-2.7/scratch/check_sys_un

Configure: Autodetecting ZMQ settings...

    Custom ZMQ dir:

************************************************

creating build/temp.linux-armv6l-2.7/scratch/tmp

cc -c /tmp/timer_create7wCR4A.c -o build/temp.linux-armv6l-2.7/scratch/tmp/timer_create7wCR4A.o

cc build/temp.linux-armv6l-2.7/scratch/tmp/timer_create7wCR4A.o -o build/temp.linux-armv6l-2.7/scratch/a.out

build/temp.linux-armv6l-2.7/scratch/tmp/timer_create7wCR4A.o: In function `main':

timer_create7wCR4A.c:(.text+0x14): undefined reference to `timer_create'

collect2: error: ld returned 1 exit status

arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -Izmq/utils -Izmq/backend/cython -Izmq/devices -c build/temp.linux-armv6l-2.7/scratch/vers.c -o build/temp.linux-armv6l-2.7/scratch/vers.o

build/temp.linux-armv6l-2.7/scratch/vers.c:4:17: fatal error: zmq.h: No such file or directory

 #include "zmq.h"

                 ^

compilation terminated.



error: command 'arm-linux-gnueabihf-gcc' failed with exit status 1



Failed with default libzmq, trying again with /usr/local

{'libraries': ['zmq'], 'extra_link_args': [], 'runtime_library_dirs': [], 'library_dirs': ['/usr/local/lib'], 'include_dirs': ['/usr/local/include']}

Configure: Autodetecting ZMQ settings...

    Custom ZMQ dir:       /usr/local

************************************************

cc -c /tmp/timer_createEh3fU0.c -o build/temp.linux-armv6l-2.7/scratch/tmp/timer_createEh3fU0.o

Assembler messages:

Fatal error: can't create build/temp.linux-armv6l-2.7/scratch/tmp/timer_createEh3fU0.o: No such file or directory

arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -I/usr/local/include -Izmq/utils -Izmq/backend/cython -Izmq/devices -c build/temp.linux-armv6l-2.7/scratch/vers.c -o build/temp.linux-armv6l-2.7/scratch/vers.o

build/temp.linux-armv6l-2.7/scratch/vers.c:4:17: fatal error: zmq.h: No such file or directory

 #include "zmq.h"

                 ^

compilation terminated.



error: command 'arm-linux-gnueabihf-gcc' failed with exit status 1



Warning: Couldn't find an acceptable libzmq on the system.



If you expected pyzmq to link against an installed libzmq, please check to make sure:



    * You have a C compiler installed

    * A development version of Python is installed (including headers)

    * A development version of ZMQ >= 3.2 is installed (including headers)

    * If ZMQ is not in a default location, supply the argument --zmq=<path>

    * If you did recently install ZMQ to a default location,

      try rebuilding the ld cache with `sudo ldconfig`

      or specify zmq's location with `--zmq=/usr/local`



You can skip all this detection/waiting nonsense if you know

you want pyzmq to bundle libzmq as an extension by passing:



    `--zmq=bundled`



I will now try to build libzmq as a Python extension

unless you interrupt me (^C) in the next 10 seconds...



************************************************

 1...

Using bundled libzmq

already have bundled/zeromq

attempting ./configure to generate platform.hpp

Warning: failed to configure libzmq:

/bin/sh: 1: ./configure: not found



staging platform.hpp from: buildutils/include_linux

checking for timer_create

************************************************

************************************************

creating build/temp.linux-armv6l-2.7/tmp

cc -c /tmp/timer_createXLUUXL.c -o build/temp.linux-armv6l-2.7/tmp/timer_createXLUUXL.o

cc build/temp.linux-armv6l-2.7/tmp/timer_createXLUUXL.o -o build/temp.linux-armv6l-2.7/a.out

build/temp.linux-armv6l-2.7/tmp/timer_createXLUUXL.o: In function `main':

timer_createXLUUXL.c:(.text+0x14): undefined reference to `timer_create'

collect2: error: ld returned 1 exit status

no timer_create, linking librt

************************************************

building 'zmq.libzmq' extension

creating build/temp.linux-armv6l-2.7/buildutils

creating build/temp.linux-armv6l-2.7/bundled

creating build/temp.linux-armv6l-2.7/bundled/zeromq

creating build/temp.linux-armv6l-2.7/bundled/zeromq/src

creating build/temp.linux-armv6l-2.7/bundled/zeromq/tweetnacl

creating build/temp.linux-armv6l-2.7/bundled/zeromq/tweetnacl/src

creating build/temp.linux-armv6l-2.7/bundled/zeromq/tweetnacl/contrib

creating build/temp.linux-armv6l-2.7/bundled/zeromq/tweetnacl/contrib/randombytes

arm-linux-gnueabihf-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fno-strict-aliasing -D_FORTIFY_SOURCE=2 -g -fstack-protector-strong -Wformat -Werror=format-security -fPIC -DZMQ_HAVE_CURVE=1 -DZMQ_USE_TWEETNACL=1 -DZMQ_USE_EPOLL=1 -Ibundled/zeromq/include -Ibundled/zeromq/tweetnacl/src -Ibundled/zeromq/tweetnacl/contrib/randombytes -Ibundled -I/usr/include/python2.7 -c buildutils/initlibzmq.c -o build/temp.linux-armv6l-2.7/buildutils/initlibzmq.o

buildutils/initlibzmq.c:10:20: fatal error: Python.h: No such file or directory

 #include "Python.h"

                    ^

compilation terminated.

error: command 'arm-linux-gnueabihf-gcc' failed with exit status 1

----------------------------------------
Cleaning up...
Command /usr/bin/python -c "import setuptools, tokenize;__file__='/tmp/pip-build-vfKpE0/pyzmq/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-wwXCX8-record/install-record.txt --single-version-externally-managed --compile failed with error code 1 in /tmp/pip-build-vfKpE0/pyzmq
Traceback (most recent call last):
  File "/usr/bin/pip", line 9, in <module>
    load_entry_point('pip==1.5.6', 'console_scripts', 'pip')()
  File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 248, in main
    return command.main(cmd_args)
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 161, in main
    text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 68: ordinal not in range(128)
Bombs out again.

Ideas?

User avatar
MrYsLab
Posts: 376
Joined: Mon Dec 15, 2014 7:14 pm
Location: Noo Joysey, USA

Re: Sharing data between python scripts through a LAN

Sat Sep 02, 2017 12:37 am

I will create a Jessie image and test, but in the meantime, if you to want to try building zeromq I am including a link to the instructions. Just substitute pip for pip3 in that last line of the instructions. Here is the link:
https://github.com/MrYsLab/razmq/wiki/Z ... structions

This will probably take about a half an hour to build (on a RPi 3).

pumpkinpi
Posts: 141
Joined: Fri Apr 18, 2014 7:11 pm

Re: Sharing data between python scripts through a LAN

Sat Sep 02, 2017 1:14 am

I think I will try a new build as well. I have a spare Pi3 here.

PPi

User avatar
MrYsLab
Posts: 376
Joined: Mon Dec 15, 2014 7:14 pm
Location: Noo Joysey, USA

Re: Sharing data between python scripts through a LAN

Sat Sep 02, 2017 1:18 am

I am seeing an issue on Jessie as well. I am going to try the whole zeromq build process and let you know if that works.

User avatar
MrYsLab
Posts: 376
Joined: Mon Dec 15, 2014 7:14 pm
Location: Noo Joysey, USA

Re: Sharing data between python scripts through a LAN

Sat Sep 02, 2017 1:43 am

It looks like pyzmq is requiring a newer version of zeromq. I will download and build that version and get back to you on status. The build instructions I provided are for an older version zeromq.

User avatar
MrYsLab
Posts: 376
Joined: Mon Dec 15, 2014 7:14 pm
Location: Noo Joysey, USA

Re: Sharing data between python scripts through a LAN

Sat Sep 02, 2017 3:09 am

Ok, I think I have it working now. Here are the steps to get things installed and working.

1. Type:

Code: Select all

sudo apt-get install python-dev
This will install python.h

2. Update numpy to latest version:

Code: Select all

sudo pip install numpy --upgrade
This is very slow. It will probably take about a half hour to update

3. There is a "bug" in pip in installing executables for python 2(this was introduced with the latest updates to pypi) and so the backplane and monitor utilities will not function if you install using pip. To get around this do the following:
a. Download the distribution from github. Go to https://github.com/MrYsLab/python_banyan and press the green download button and unzip the file.
b. go into the directory python_banyan-master and make sure you see setup.py
c. type:

Code: Select all

sudo setup.py install
Please let me know if you still have issues.

pumpkinpi
Posts: 141
Joined: Fri Apr 18, 2014 7:11 pm

Re: Sharing data between python scripts through a LAN

Sat Sep 02, 2017 3:18 am

Ok! Will do. Thanks.

asandford
Posts: 1997
Joined: Mon Dec 31, 2012 12:54 pm
Location: Waterlooville

Re: Sharing data between python scripts through a LAN

Sun Sep 03, 2017 12:21 am

MrYsLab wrote:
Fri Sep 01, 2017 12:37 pm
And streaming messages on a single Rpi 3 is twice as fast as Mosquitto MQTT (test code is available in github).
As there is no mention of how this test was run, the results are meaningless.

Encryption and authentication?

Last will and testament?

QoS?

Support for Javascript / Java / C / C++ / Pascal / <pick a language>

Can I run it on my Arduino?

If the answer is "No" to any of the above, then it isn't a replacement for MQTT

Also, if you remove the following from the mqpub (as it isn't in the banyan_pub), what are the results then?

Code: Select all

packed = umsgpack.packb(payload)  <== why? 
And this :

Code: Select all

if x == 45000:
                timex = time.asctime(time.localtime(time.time()))
                payload2 = {'modulo': x, 'time': timex}
                packed2 = umsgpack.packb(payload2)
                self.publish("m2", packed2)
                print('m2 message: ' + str(x) + ' ' + timex)

If it isn't tested every loop cycle, then it's pointless. If it is then you've hobbled the code as you don't perform the same test in the banyan code.

[edit]

The subscribe code is even more devious:

The depack adds more time:

Code: Select all

mg = umsgpack.unpackb(msg.payload)
data = mg['msg']
And why is it testing for things on another topic?:

Code: Select all

self.subscribe("m2", 0)
And then testing messages?

Code: Select all

if msg.topic == 'test':
...
elif msg.topic == 'm2':
            mg = umsgpack.unpackb(msg.payload)
            if mg['modulo'] == 45000:
print(msg.payload)
None of that appears in your banyan code.

Nice way to rig rsults.

[edit 2] In case things change:

Code: Select all

banyan_sub.py
import sys
import time
from python_banyan.banyan_base import BanyanBase


class BanyanSub(BanyanBase):
    """
    This class subscribes to the "test" topic, and sits in a loop
    receiving messages. When 100000 messages have been received,
    it shuts down the loop and prints both the elapsed time taken
    to receive all messages as well as the current time.
    """
    def __init__(self):
        """
        Subscribe to the "test" topic, initialize the message count
        and enter the receive loop.
        """
        super(BanyanSub, self).__init__(process_name='Banyan Subscriber')
        # allow time for connections to be established
        time.sleep(.3)

        # the start time
        self.start = None

        # the end time
        self.end = None

        print('Expecting 100000 messages.')
        self.set_subscriber_topic('test')
        self.message_count = 0
        self.receive_loop()

    def incoming_message_processing(self, topic, payload):
        """
        When a message is received, extract the message number, bump the
        message count by 1.

        On the first message, start the elapsed time timer.

        On the last message determine the elapsed time, and print it
        and the current time. Finally exit
        :param topic: Message Topic string
        :param payload: Message Data
        :return:
        """
        data = payload['msg']
        self.message_count += 1

        # time.sleep(.001)
        # if not self.message_count % 100:
        #     print(self.message_count)

        if data == 0:
            self.start = time.time()

        if data == 99999:
            self.end = time.time()
            localtime = time.asctime(time.localtime(time.time()))
            print('Task completed at: ', localtime)
            print('{} Total messages received in {} seconds.'.format(self.message_count, self.end - self.start))
            super(BanyanSub, self).clean_up()
            sys.exit(0)

# instantiate this class
BanyanSub()

Code: Select all

banyan pub
import sys
import time
from python_banyan.banyan_base import BanyanBase


class BanyanPub(BanyanBase):
    """
    This class publishes 100000 "test" topic messages as fast as it can.
    It then prints the current time and exits.
    """

    def __init__(self):
        super(BanyanPub, self).__init__(process_name='Banyan publisher')

        print('Publishing 100000 messages.')
        time.sleep(.3)

        for x in range(0, 100000):
            payload = {'msg': x}
            self.publish_payload(payload, 'test')

        localtime = time.asctime(time.localtime(time.time()))

        print('Task completed at: ', localtime)

        super(BanyanPub, self).clean_up()
        sys.exit(0)

# instantiate this class
BanyanPub()
[code]mqtt sub
import sys
import time

import paho.mqtt.client as mqtt
import umsgpack


class MQSUB(mqtt.Client):
    """
    This class subscribes to the "test" topic, and sits in a loop
    receiving messages. When 100000 messages have been received,
    it shuts down the loop and prints both the elapsed time taken
    to receive all messages as well as the current time.
    """

    def __init__(self):
        """
        Start the client, subscribe to "test" and start the loop
        """
        super(MQSUB, self).__init__()
        # super().on_connect = self.on_connect
        # super().on_message = self.on_message

        self.start = 0
        self.end = 0
        self.message_count = 0

        self.connect('192.168.2.192', 1883, 60)
        self.subscribe("test", 0)
        self.subscribe("m2", 0)


        self.loop_forever()

    def on_connect(self, mqttc, obj, flags, rc):
        """
        Let the use know we are connected
        :param mqttc: unused
        :param obj: unused
        :param flags: unused
        :param rc: unused
        :return: unused
        """
        print('MQSUB Connected - Expecting 100000 Messages')

    def on_message(self, mqttc, obj, msg):
        """

        :param mqttc: the client
        :param obj: unused
        :param msg: the message containing the payload
        :return:
        """
        # unpack the messagepack message
        if msg.topic == 'test':
            mg = umsgpack.unpackb(msg.payload)
            data = mg['msg']

            # update the count of messages received and start the
            # elapsed time timer
            self.message_count += 1
            if data == 0:
                self.start = time.time()

            # wait for the last message, then print out the current time
            # and an elapsed time message.
            # finally stop the loop and exit.
            if data == 99999:
                self.end = time.time()
                localtime = time.asctime(time.localtime(time.time()))

                print('Task completed on: ', localtime)
                print('{} Total messages received in {} seconds.'.format(self.message_count,
                                                                         self.end - self.start))
                time.sleep(1)
                self.loop_stop()
                sys.exit(0)
        elif msg.topic == 'm2':
            mg = umsgpack.unpackb(msg.payload)
            if mg['modulo'] == 45000:
                print(msg.payload)
        else:
            print('unkown topic' + msg.topic)


MQSUB()
[code][code]
mqtt pub
import paho.mqtt.client as mqtt
import sys
import time
import umsgpack


class MQPUB(mqtt.Client):
    """
        This class publishes 100000 "test" topic messages as fast as it can.
        It then prints the current time and exits.
        """
    def __init__(self):
        """
        This method connects, starts the loop and then publishes
        all the messages until complete.

        It then prints the current time before stopping the loop and exiting.
        """
        super(MQPUB, self).__init__()

        self.connect('192.168.2.192', 1883, 60)
        print("MQPUB Connected - Sending 100000 messages")

        time.sleep(2)

        for x in range(0, 100000):
            payload = {'msg': x}
            packed = umsgpack.packb(payload)
            self.publish("test", packed)

            if x == 45000:
                timex = time.asctime(time.localtime(time.time()))
                payload2 = {'modulo': x, 'time': timex}
                packed2 = umsgpack.packb(payload2)
                self.publish("m2", packed2)
                print('m2 message: ' + str(x) + ' ' + timex)


        localtime = time.asctime(time.localtime(time.time()))

        print('Task completed on: ', localtime)
        time.sleep(1)
        self.loop_stop()
        sys.exit(0)



# instantiate the class
MQPUB()
Not a level playing field is it?
Last edited by asandford on Sun Sep 03, 2017 12:56 am, edited 2 times in total.

pumpkinpi
Posts: 141
Joined: Fri Apr 18, 2014 7:11 pm

Re: Sharing data between python scripts through a LAN

Sun Sep 03, 2017 12:25 am

I think you meant:

Code: Select all

sudo python setup.py install
I tried that and got:

Code: Select all

zip_safe flag not set; analyzing archive contents...
creating dist
creating 'dist/python_banyan-2.1-py2.7.egg' and adding 'build/bdist.linux-armv6l/egg' to it
removing 'build/bdist.linux-armv6l/egg' (and everything under it)
Processing python_banyan-2.1-py2.7.egg
Copying python_banyan-2.1-py2.7.egg to /usr/local/lib/python2.7/dist-packages
Adding python-banyan 2.1 to easy-install.pth file
Installing backplane script to /usr/local/bin
Installing monitor script to /usr/local/bin

Installed /usr/local/lib/python2.7/dist-packages/python_banyan-2.1-py2.7.egg
Processing dependencies for python-banyan==2.1
Searching for psutil
Reading https://pypi.python.org/simple/psutil/
Best match: psutil 5.3.0
Downloading https://pypi.python.org/packages/1c/da/555e3ad3cad30f30bcf0d539cdeae5c8e7ef9e2a6078af645c70aa81e418/psutil-5.3.0.tar.gz#md5=2ff7b2030b3bb786594d6276f32b6699
Processing psutil-5.3.0.tar.gz
Writing /tmp/easy_install-N4RB8T/psutil-5.3.0/setup.cfg
Running psutil-5.3.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-N4RB8T/psutil-5.3.0/egg-dist-tmp-dEIkFJ
psutil/_psutil_common.c:9:20: fatal error: Python.h: No such file or directory
 #include <Python.h>
                    ^
compilation terminated.
error: Setup script exited with error: command 'arm-linux-gnueabihf-gcc' failed with exit status 1
So, randomly I tried:

Code: Select all

sudo python3 setup.py install
It got further, but spat out this:

Code: Select all

Writing /tmp/easy_install-xxxae3li/numpy-1.13.1/setup.cfg
Running numpy-1.13.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-xxxae3li/numpy-1.13.1/egg-dist-tmp-iboxq3q7
Running from numpy source directory.
/tmp/easy_install-xxxae3li/numpy-1.13.1/setup.py:367: UserWarning: Unrecognized setuptools command, proceeding with generating Cython sources and expanding templates
  run_build = parse_setuppy_commands()
Warning: Can't read registry to find the necessary compiler setting
Make sure that Python modules winreg, win32api or win32con are installed.
/usr/lib/python3.4/distutils/dist.py:260: UserWarning: Unknown distribution option: 'python_requires'
  warnings.warn(msg)
Warning: distutils distribution has been initialized, it may be too late to add a subpackage compatWarning: distutils distribution has been initialized, it may be too late to add an extension _dummyWarning: distutils distribution has been initialized, it may be too late to add a library npysort/tmp/easy_install-xxxae3li/numpy-1.13.1/numpy/distutils/system_info.py:572: UserWarning: 
    Atlas (http://math-atlas.sourceforge.net/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [atlas]) or by setting
    the ATLAS environment variable.
  self.calc_info()
/tmp/easy_install-xxxae3li/numpy-1.13.1/numpy/distutils/system_info.py:572: UserWarning: 
    Blas (http://www.netlib.org/blas/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [blas]) or by setting
    the BLAS environment variable.
  self.calc_info()
/tmp/easy_install-xxxae3li/numpy-1.13.1/numpy/distutils/system_info.py:572: UserWarning: 
    Blas (http://www.netlib.org/blas/) sources not found.
    Directories to search for the sources can be specified in the
    numpy/distutils/site.cfg file (section [blas_src]) or by setting
    the BLAS_SRC environment variable.
  self.calc_info()
/bin/sh: 1: svnversion: not found
non-existing path in 'numpy/distutils': 'site.cfg'
/bin/sh: 1: svnversion: not found
Warning: distutils distribution has been initialized, it may be too late to add an extension multiarrayWarning: distutils distribution has been initialized, it may be too late to add an extension umathWarning: distutils distribution has been initialized, it may be too late to add an extension umath_testsWarning: distutils distribution has been initialized, it may be too late to add an extension test_rationalWarning: distutils distribution has been initialized, it may be too late to add an extension struct_ufunc_testWarning: distutils distribution has been initialized, it may be too late to add an extension multiarray_testsWarning: distutils distribution has been initialized, it may be too late to add an extension operand_flag_testsWarning: distutils distribution has been initialized, it may be too late to add a subpackage coreWarning: distutils distribution has been initialized, it may be too late to add a subpackage commandWarning: distutils distribution has been initialized, it may be too late to add a subpackage fcompilerWarning: distutils distribution has been initialized, it may be too late to add a subpackage distutilsWarning: distutils distribution has been initialized, it may be too late to add a subpackage docWarning: distutils distribution has been initialized, it may be too late to add a subpackage f2pyWarning: distutils distribution has been initialized, it may be too late to add an extension fftpack_liteWarning: distutils distribution has been initialized, it may be too late to add a subpackage fftWarning: distutils distribution has been initialized, it may be too late to add a subpackage lib/tmp/easy_install-xxxae3li/numpy-1.13.1/numpy/distutils/system_info.py:572: UserWarning: 
    Lapack (http://www.netlib.org/lapack/) libraries not found.
    Directories to search for the libraries can be specified in the
    numpy/distutils/site.cfg file (section [lapack]) or by setting
    the LAPACK environment variable.
  self.calc_info()
/tmp/easy_install-xxxae3li/numpy-1.13.1/numpy/distutils/system_info.py:572: UserWarning: 
    Lapack (http://www.netlib.org/lapack/) sources not found.
    Directories to search for the sources can be specified in the
    numpy/distutils/site.cfg file (section [lapack_src]) or by setting
    the LAPACK_SRC environment variable.
  self.calc_info()
Warning: distutils distribution has been initialized, it may be too late to add an extension lapack_liteWarning: distutils distribution has been initialized, it may be too late to add an extension _umath_linalgWarning: distutils distribution has been initialized, it may be too late to add a subpackage linalgWarning: distutils distribution has been initialized, it may be too late to add a subpackage maWarning: distutils distribution has been initialized, it may be too late to add a subpackage matrixlibWarning: distutils distribution has been initialized, it may be too late to add a subpackage polynomialWarning: distutils distribution has been initialized, it may be too late to add an extension mtrandWarning: distutils distribution has been initialized, it may be too late to add a subpackage randomWarning: distutils distribution has been initialized, it may be too late to add a subpackage testingWarning: distutils distribution has been initialized, it may be too late to add a subpackage numpy/usr/lib/python3.4/distutils/dist.py:260: UserWarning: Unknown distribution option: 'python_requires'
  warnings.warn(msg)
Could not locate executable gfortran
Could not locate executable f95
Could not locate executable ifort
Could not locate executable ifc
Could not locate executable lf95
Could not locate executable pgfortran
Could not locate executable f90
Could not locate executable f77
Could not locate executable fort
Could not locate executable efort
Could not locate executable efc
Could not locate executable g77
Could not locate executable g95
Could not locate executable pathf95
don't know how to compile Fortran code on platform 'posix'
_configtest.c:1:5: warning: conflicting types for built-in function ‘exp’
 int exp (void);
     ^
_configtest.o: In function `main':
/tmp/easy_install-xxxae3li/numpy-1.13.1/_configtest.c:6: undefined reference to `exp'
collect2: error: ld returned 1 exit status
_configtest.o: In function `main':
/tmp/easy_install-xxxae3li/numpy-1.13.1/_configtest.c:6: undefined reference to `exp'
collect2: error: ld returned 1 exit status
_configtest.c:1:5: warning: conflicting types for built-in function ‘exp’
 int exp (void);
     ^
_configtest.c:1:24: fatal error: sys/endian.h: No such file or directory
 #include <sys/endian.h>
                        ^
compilation terminated.
_configtest.c:1:24: fatal error: sys/endian.h: No such file or directory
 #include <sys/endian.h>
                        ^
compilation terminated.
_configtest.c: In function ‘main’:
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) >= 0)];
                ^
_configtest.c: In function ‘main’:
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) == 4)];
                ^
_configtest.c: In function ‘main’:
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) >= 0)];
                ^
_configtest.c: In function ‘main’:
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) == 8)];
                ^
_configtest.c: In function ‘main’:
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) >= 0)];
                ^
_configtest.c: In function ‘main’:
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) == 8)];
                ^
_configtest.c: In function ‘main’:
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) >= 0)];
                ^
_configtest.c: In function ‘main’:
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) == 16)];
                ^
_configtest.c: In function ‘main’:
_configtest.c:7:12: error: ‘SIZEOF_LONGDOUBLE’ undeclared (first use in this function)
     (void) SIZEOF_LONGDOUBLE;
            ^
_configtest.c:7:12: note: each undeclared identifier is reported only once for each function it appears in
_configtest.c: In function ‘main’:
_configtest.c:7:12: error: ‘SIZEOF_LONGDOUBLE’ undeclared (first use in this function)
     (void) SIZEOF_LONGDOUBLE;
            ^
_configtest.c:7:12: note: each undeclared identifier is reported only once for each function it appears in
_configtest.c: In function ‘main’:
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) >= 0)];
                ^
_configtest.c: In function ‘main’:
_configtest.c:5:16: error: size of array ‘test_array’ is negative
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) == 16)];
                ^
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
_configtest.c: In function ‘main’:
_configtest.c:5:16: error: size of array ‘test_array’ is negative
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) == 16)];
                ^
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
_configtest.c: In function ‘main’:
_configtest.c:5:16: error: size of array ‘test_array’ is negative
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) == 12)];
                ^
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
_configtest.c: In function ‘main’:
_configtest.c:5:16: error: size of array ‘test_array’ is negative
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) == 12)];
                ^
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
_configtest.c: In function ‘main’:
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) == 8)];
                ^
_configtest.c: In function ‘main’:
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) >= 0)];
                ^
_configtest.c: In function ‘main’:
_configtest.c:5:16: error: size of array ‘test_array’ is negative
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) == 32)];
                ^
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
_configtest.c: In function ‘main’:
_configtest.c:5:16: error: size of array ‘test_array’ is negative
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) == 32)];
                ^
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
_configtest.c: In function ‘main’:
_configtest.c:5:16: error: size of array ‘test_array’ is negative
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) == 24)];
                ^
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
_configtest.c: In function ‘main’:
_configtest.c:5:16: error: size of array ‘test_array’ is negative
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) == 24)];
                ^
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
_configtest.c: In function ‘main’:
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) == 16)];
                ^
_configtest.c: In function ‘main’:
_configtest.c:7:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) >= 0)];
                ^
_configtest.c: In function ‘main’:
_configtest.c:7:16: error: size of array ‘test_array’ is negative
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) == 8)];
                ^
_configtest.c:7:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
_configtest.c: In function ‘main’:
_configtest.c:7:16: error: size of array ‘test_array’ is negative
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) == 8)];
                ^
_configtest.c:7:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
_configtest.c: In function ‘main’:
_configtest.c:7:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) == 4)];
                ^
_configtest.c: In function ‘main’:
_configtest.c:7:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) >= 0)];
                ^
_configtest.c: In function ‘main’:
_configtest.c:7:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) == 8)];
                ^
_configtest.c: In function ‘main’:
_configtest.c:7:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) >= 0)];
                ^
_configtest.c: In function ‘main’:
_configtest.c:7:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) == 8)];
                ^
_configtest.c: In function ‘main’:
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) >= 0)];
                ^
_configtest.c: In function ‘main’:
_configtest.c:5:16: warning: variable ‘test_array’ set but not used [-Wunused-but-set-variable]
     static int test_array [1 - 2 * !(((long) (sizeof (npy_check_sizeof_type))) == 8)];
                ^
_configtest.c:1:5: warning: conflicting types for built-in function ‘exp’
 int exp (void);
     ^
_configtest.o: In function `main':
/tmp/easy_install-xxxae3li/numpy-1.13.1/_configtest.c:6: undefined reference to `exp'
collect2: error: ld returned 1 exit status
_configtest.o: In function `main':
/tmp/easy_install-xxxae3li/numpy-1.13.1/_configtest.c:6: undefined reference to `exp'
collect2: error: ld returned 1 exit status
_configtest.c:1:5: warning: conflicting types for built-in function ‘exp’
 int exp (void);
     ^
_configtest.c:1:5: warning: conflicting types for built-in function ‘floor’
 int floor (void);
     ^
_configtest.c:2:5: warning: conflicting types for built-in function ‘ldexp’
 int ldexp (void);
     ^
_configtest.c:3:5: warning: conflicting types for built-in function ‘atan’
 int atan (void);
     ^
_configtest.c:4:5: warning: conflicting types for built-in function ‘fabs’
 int fabs (void);
     ^
_configtest.c:5:5: warning: conflicting types for built-in function ‘acos’
 int acos (void);
     ^
_configtest.c:6:5: warning: conflicting types for built-in function ‘tanh’
 int tanh (void);
     ^
_configtest.c:7:5: warning: conflicting types for built-in function ‘log10’
 int log10 (void);
     ^
_configtest.c:8:5: warning: conflicting types for built-in function ‘fmod’
 int fmod (void);
     ^
_configtest.c:9:5: warning: conflicting types for built-in function ‘sinh’
 int sinh (void);
     ^
_configtest.c:10:5: warning: conflicting types for built-in function ‘ceil’
 int ceil (void);
     ^
_configtest.c:11:5: warning: conflicting types for built-in function ‘frexp’
 int frexp (void);
     ^
_configtest.c:12:5: warning: conflicting types for built-in function ‘modf’
 int modf (void);
     ^
_configtest.c:13:5: warning: conflicting types for built-in function ‘tan’
 int tan (void);
     ^
_configtest.c:14:5: warning: conflicting types for built-in function ‘asin’
 int asin (void);
     ^
_configtest.c:15:5: warning: conflicting types for built-in function ‘cos’
 int cos (void);
     ^
_configtest.c:16:5: warning: conflicting types for built-in function ‘cosh’
 int cosh (void);
     ^
_configtest.c:17:5: warning: conflicting types for built-in function ‘exp’
 int exp (void);
     ^
_configtest.c:18:5: warning: conflicting types for built-in function ‘log’
 int log (void);
     ^
_configtest.c:19:5: warning: conflicting types for built-in function ‘sqrt’
 int sqrt (void);
     ^
_configtest.c:20:5: warning: conflicting types for built-in function ‘sin’
 int sin (void);
     ^
Kept going with other errors.

No dice yet.

asandford
Posts: 1997
Joined: Mon Dec 31, 2012 12:54 pm
Location: Waterlooville

Re: Sharing data between python scripts through a LAN

Sun Sep 03, 2017 1:01 am

pumpkinpi wrote:
Sun Sep 03, 2017 12:25 am
Kept going with other errors.

No dice yet.
Or you could just use the industry standard which is supported by just about everything.

User avatar
MrYsLab
Posts: 376
Joined: Mon Dec 15, 2014 7:14 pm
Location: Noo Joysey, USA

Re: Sharing data between python scripts through a LAN

Sun Sep 03, 2017 2:06 am

I just did installs for both python 2 and 3 on a clean jessie image.

1. I installed jessie 2017-07-05 on an sd card.
2. I am running headless and ssh'ed into the PI.
3. I ran raspi-config and expanded memory and set the boot options for Desktop/CLI to desktop gui requiring login.
4. I rebooted the Rpi.
5. I did a sudo apt-get update and sudo apt-get upgrade.

Installing for python 3:
1. I did a sudo pip3 install python-banyan
This built several of the packages and install took about 20 mins.

Installing for python 2:
1. I uninstalled for python 3 using sudo pip3 uninstall python-banyan.
2. I did a sudo apt-get install python-dev.
3.Download this file:
https://github.com/MrYsLab/python_banya ... ne-any.whl
4. Type:

Code: Select all

sudo pip install ./python_banyan-2.1-py2-none-any.whl
Again this will take about 20 minutes or so for all the required packages to compile.

In both cases the installations were successful.

Again, let me know if this works.

pumpkinpi
Posts: 141
Joined: Fri Apr 18, 2014 7:11 pm

Re: Sharing data between python scripts through a LAN

Sun Sep 03, 2017 10:52 pm

I misread your instructions. Will try again.

pumpkinpi
Posts: 141
Joined: Fri Apr 18, 2014 7:11 pm

Re: Sharing data between python scripts through a LAN

Mon Sep 04, 2017 3:48 am

In the meantime, I tried asandford's recommendation. I did this:

Code: Select all

sudo apt-get install mosquitto
sudo pip install paho-mqtt
And these two things installed no probs.

I did some reading here:
https://pypi.python.org/pypi/paho-mqtt/1.3.0

but quickly got lost. Have not yet found a tutorial for dummies on how to make this work on my raspberry pi's running simple python scripts. I am just looking to pass strings and simple numbers between them.

If anybody knows of guides, I would appreciate direction.

I think I just need to see a few examples. To late to keep googling...

PPi.

mattmiller
Posts: 2113
Joined: Thu Feb 05, 2015 11:25 pm

Re: Sharing data between python scripts through a LAN

Mon Sep 04, 2017 6:44 am

I found this one got me started
http://www.hivemq.com/blog/mqtt-client- ... aho-python

Just to note - MQTT is an established protocol and can be run on practically any IoT device (pi/Arduino/ESP8266 etc) so learning how to implement it is well worth it

User avatar
DougieLawson
Posts: 36322
Joined: Sun Jun 16, 2013 11:19 pm
Location: Basingstoke, UK
Contact: Website Twitter

Re: Sharing data between python scripts through a LAN

Mon Sep 04, 2017 4:47 pm

Learn MQTT is a few hours with http://www.hivemq.com/mqtt-essentials/
Note: Having anything humorous in your signature is completely banned on this forum. Wear a tin-foil hat and you'll get a ban.

Any DMs sent on Twitter will be answered next month.

This is a doctor free zone.

Return to “Python”