cbrown330
Posts: 13
Joined: Sat Jan 26, 2013 6:16 am

only for the greats in socket programming

Sat Feb 23, 2013 2:51 am

ok first thing is here is my code:

import socket
import sys
from thread import *

host = ''
port = 1234

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'Socket Created'

try:

s.bind((host, port))
except socket.error , msg:
print 'Bind failed. Error Code : ' +str(msg[0]) + ' Message ' + msg[1]
sys.exit()
print 'Socket bind complete'

s.listen(10)
print 'socket now Listening'

def clientthread(conn):
conn.send('Wlecome to the Server. Type somthing and hit enter\r')


while True:

data = conn.recv(1024)
print data
reply = 'OK.....' + data
if not data:
break

conn.send('test\r')

conn.close()

while 1:
conn, addr = s.accept()
print 'connected with ' + addr[0] + ':' + str(addr[1])

start_new_thread(clientthread ,(conn,))

s.close()


ok so this code works, however it is not giving me the results from a certain socket that i am looking for. Meaning i can run this script and connect with a remote host with putty or DOS telnet, and what ever I type in is reported to the script just fine
ie. If i type "this sucks" then i get the same string text to the script just as i wrote it.

Now i have a HMI machine that i am also wanting to send string data to and from the script. the HMI is using a raw TCP/IP active socket to connect to the script. When i send data to the HMI its golden however when i get the string from the HMI i get some wierd results.
ie send "this sucks" using a variable from the HMI i may get

T
his sucks

this is what the script reads. Now i know everyone will be saying its the HMI. I can Telnet to the HMI and when i send the string to the session i Get the string in one line. I will need this string that is coming from the HMI to be one line since I will be taking this string and decoding the information from it. If anyone read through this whole long post and would like to shed some light it would be awasome.

User avatar
bgreat
Posts: 235
Joined: Mon Jan 23, 2012 2:09 pm

Re: only for the greats in socket programming

Sat Feb 23, 2013 9:51 am

Please post your source inside using the "Code" button to preserve formatting. This makes it possible for others to review your code with the indenting as you entered.

If you are connecting as a client to a telnet daemon on the "HMI" system, it is possible you are seeing the effect of the embedded telnet protocol characters. You can confirm this by displaying the character codes of all received data in addition to the character strings. If you would like more precise help, you will need to post the rest of your code.

Enjoy!
Bill

User avatar
rurwin
Forum Moderator
Forum Moderator
Posts: 4258
Joined: Mon Jan 09, 2012 3:16 pm
Contact: Website

Re: only for the greats in socket programming

Sat Feb 23, 2013 10:29 am

There is nothing in the TCP spec that says a message has to be sent in one packet, rather the reverse. Your HMI seems to be sending two packets, one containing the initial "T" and one containing the rest.

It is important to treat the network connection as a stream of bytes, not as a stream of packets, and to buffer them up until you get a complete message. It is not an error to have a delay part-way through a message, or to have a message split into many packets. The TCP stack itself is responsible for splitting messages into packets and it may delay packets, so although your client sends everything at the same time, there may be a delay in the middle of the message due to buffering or network problems.

It is generally not sufficient to wait until no more data is available, because the data may be on their way but are still buffered on the sending machine. You need a timeout of, say, between a second and a minute depending on how long the path between the machines is, and on how long you can afford.

There are generally heuristics in the TCP stack to determine when to send buffered data, but they are not standard. Generally ending a message with a new-line character will get it sent out immediately, but not always. You can also flush(), but even that is not certain.

cbrown330
Posts: 13
Joined: Sat Jan 26, 2013 6:16 am

Re: only for the greats in socket programming

Sat Feb 23, 2013 5:45 pm

Ok I understand what your saying and I would almost bet that your right since I could open a dual socket to the script and while one was send ing data the other one would wait till the data was sent. When this happen I did recieve the full string from the HMI on one line. So here is a question, At the point where i am printing out the data variable is within the while loop. If I waited till the loop was doen and then printed our the data variable outside the while loop do you like it would be in one string line or would I only get the last set of bits taht came accross? Or how can i program a delay in the loop to ensure I get the full string. Now i had sent a \r in the message to see if I can get the carr return, I got the same response. The HMI requieres a \r at the end of the string so that it will display on the screen correctly.

cbrown330
Posts: 13
Joined: Sat Jan 26, 2013 6:16 am

Re: only for the greats in socket programming

Sat Feb 23, 2013 5:46 pm

Code: Select all

import socket
import sys
from thread import *

host = ''
port = 1234

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'Socket Created'

try:
	
	s.bind((host, port))
except socket.error , msg:
	print 'Bind failed. Error Code : ' +str(msg[0]) + ' Message ' + msg[1]
	sys.exit()
print 'Socket bind complete'

s.listen(10)
print 'socket now Listening'

def clientthread(conn):
	conn.send('Wlecome to the Server. Type somthing and hit enter\r')


	while True:

		data = conn.recv(1024)
		print data
		reply = 'OK.....' + data
		if not data:
	   	   break

		conn.send('test\r')

	conn.close()

while 1:
	conn, addr = s.accept()
	print 'connected with ' + addr[0] + ':' + str(addr[1])

	start_new_thread(clientthread ,(conn,))

s.close()

-rst-
Posts: 1316
Joined: Thu Nov 01, 2012 12:12 pm
Location: Dublin, Ireland

Re: only for the greats in socket programming

Mon Feb 25, 2013 4:19 pm

cbrown330 wrote:Or how can i program a delay in the loop to ensure I get the full string. Now i had sent a \r in the message to see if I can get the carr return, I got the same response. The HMI requieres a \r at the end of the string so that it will display on the screen correctly.
I would assume the HMI also sends the \r at the and of messages it sends, so you should most likely read the input and buffer the read strings (msg = msg + readpart) until \r received and only then print the message.
http://raspberrycompote.blogspot.com/ - Low-level graphics and 'Coding Gold Dust'

User avatar
rurwin
Forum Moderator
Forum Moderator
Posts: 4258
Joined: Mon Jan 09, 2012 3:16 pm
Contact: Website

Re: only for the greats in socket programming

Mon Feb 25, 2013 4:46 pm

To do both the buffering and the timeouts can get complex. Here's one I wrote earlier.

Code: Select all

import logging
import socket

ALLOK = 0
TIMEOUT = 1
EOF = 2
NOTCONNECTED = 3
NEXT = 4			# next code for subclass.

hex_mode = False

def visible_char(c):
	"""Convert the given character to a visible representation.
	
	If the character is printable, just return it. Otherwise return its ordinal
	value in square brackets.
	"""
	if hex_mode:
		return " {0:02X}".format(ord(c))
	elif 31 < ord(c) < 126:
		return c
	else:
		return "["+str(ord(c))+"]"
		
def visible_string(str):
	"""Convert the given string to one in which all characters are visible.
	
	Any non-printable characters in the string are replaced by their ordinal
	value in square brackets.
	"""
	ret=""
	for c in str:
		ret = ret + visible_char(c)
	return ret
	
class GenericComms(object):
	"""Implements a generic interface to TCP and UDP.
	
	The interface is getcharacter/putcharacter/flush
	based. These functions must be efficient. Received characters are logged when
	instructed or when a character is sent. Sent characters are logged on a flush,
	and are only sent on a flush. All I/O is expected to be blocking.
	
	Subclasses define _send_string and _receive_string to interface to the underlying comms.
	"""
	
	def __init__(self, logger, logtag, rxtag="Rx:", txtag="Tx:"):
		"""Simple initialiser that just writes the instance state.
		"""
		self.logger = logger
		self.logtag = logtag
		self.rxtag = rxtag
		self.txtag = txtag
		self.rxstr = ""
		self.rxlog = ""
		self.txstr = ""
		
	def log_rx(self):
		"""Log received data.
		
		This is usually done automatically as soon as we send a character, so
		the only reason to call this function is when we are not going to send a reply.
		"""
		if len(self.rxlog) > 0:
			if self.logger != None:
				self.logger.log(logging.COMMS, self.rxtag+visible_string(self.rxlog))
			self.rxlog = ""
	
		
	def put_ch(self, c):
		"""Adds a character to those ready to be sent.
		
		If there are received characters to be logged, then it logs them.
		"""
		self.log_rx()
		self.txstr = self.txstr + c
		
	def flush(self):
		"""Flushes and logs the transmitted characters.
		"""
		if self.logger != None:
			self.logger.log(logging.COMMS, self.txtag+visible_string(self.txstr))
		self._send_string(self.txstr)
		self.txstr = ""
		
	def get_ch(self,timeout=0):
		"""Returns a single character read from the stream and an error flag.
		
		If there are no characters in the buffer, a new string is read from the
		stream, with the given timeout (default=0=no timeout).
		
		If no characters are read, then an empty string is returned along with
		the reason code returned when reading the string.
		"""
		if len(self.rxstr) == 0:
			self.rxstr, self.reason = self._receive_string(timeout)
		if len(self.rxstr) == 0:
			return "", self.reason
		c = self.rxstr[0:1]
		self.rxstr = self.rxstr[1:]
		self.rxlog = self.rxlog + c
		return c,ALLOK
		
	def _send_string(self):
		"""This function must be overridden by a subclass.
		"""
		raise NotImplementedError, "_send_string not overridden, or generic_comms not subclassed"
		
	def _receive_string(self, timeout):
		"""This function must be overridden by a subclass.
		"""
		raise NotImplementedError, "_receive_string not overridden, or generic_comms not subclassed"
		return "", EOF
		
		
class TcpComms(GenericComms):
	"""Implements a TCP server stream.
	"""
	def __init__(self, logger, logtag, port, addr="127.0.0.1", rxtag="Rx:", txtag="Tx:"):
		GenericComms.__init__(self, logger, logtag, rxtag, txtag)
		self.local_addr = (addr, port)
		self.listen = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
		self.listen.bind(self.local_addr)
		self.listen.listen(1)
		self.connected = False
		
	def getconnected(self,timeout=0):
		"""If the socket is not yet connected then we wait until it is.
	
		There is a timeout involved because for receive we may need one.
		Usually though the first character of a message will be received with
		no timeout, so it is not likely to affect us.
		"""
		if not self.connected:
			print "Trying to connect..."
			if timeout > 0:
				self.listen.settimeout(timeout)
			try:
				self.conn, self.remote_addr = self.listen.accept()
			except socket.timeout:
				print "Timed out"
				return False
			else:
				print "Connected"
				self.connected = True
		return True
		
	def _receive_string(self, timeout=0):
		"""Receive a new string from the stream.
		
		If the stream is not yet connected then we wait until it is. The same
		timeout is used for that, so we may end up waiting for two timeout periods.
		This is not likely to be a problem. Especially since the timeout is not
		likely to be used on the first character (it is there to allow
		intra-message timeouts in case we get out of sync.
		"""
		if not self.getconnected(timeout):
			return "",TIMEOUT
		
		if timeout > 0:
			self.conn.settimeout(timeout)
			
		try:
			s = self.conn.recv(1024)
		except socket.error:
			self.connected = False
			print "EOF"
			return "", EOF
		except socket.timeout:
			print "timeout"
			return "", TIMEOUT
		
		if len(s) == 0:
			self.connected = False
			print "eof"
			return "", EOF
			
		return s,ALLOK
		
	def _send_string(self, s):
		"""Send a string to the stream.
		
		We wait first until a connection is made. There is no timeout here, so
		we may wait forever.
		"""
		self.getconnected(0.1)
		if self.connected:
			try:
				self.conn.send(s)
			except socket.error:
				self.connected = False
class UdpComms(GenericComms):
	"""Implements a UDP server stream.
	"""
	def __init__(self, logger, logtag, port, addr="127.0.0.1", rxtag="Rx:", txtag="Tx:"):
		GenericComms.__init__(self, logger, logtag, rxtag, txtag)
		self.local_addr = (addr, port)
		self.conn = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
		self.conn.bind(self.local_addr)
		self.connected = True
		
	def getconnected(self,timeout=0):
		"""If the socket is not yet connected then we wait until it is.
	
		This is UDP, we don't connect.
		"""
		self.connected = True
		return True
		
	def _receive_string(self, timeout=0):
		"""Receive a new string from the stream.
		
		If the stream is not yet connected then we wait until it is. The same
		timeout is used for that, so we may end up waiting for two timeout periods.
		This is not likely to be a problem. Especially since the timeout is not
		likely to be used on the first character (it is there to allow
		intra-message timeouts in case we get out of sync.
		"""
		if not self.getconnected(timeout):
			return "",TIMEOUT
		
		if timeout > 0:
			self.conn.settimeout(timeout)
			
		try:
			s, self.remoteaddr = self.conn.recvfrom(1024)
		except socket.error:
			self.connected = False
			print "EOF"
			return "", EOF
		except socket.timeout:
			print "timeout"
			return "", TIMEOUT
		
		if len(s) == 0:
			self.connected = False
			print "eof"
			return "", EOF
			
		for c in s:
			print " (0:02x)".format(ord(c))
		print
			
		return s,ALLOK
		
	def _send_string(self, s):
		"""Send a string to the stream.
		
		We wait first until a connection is made. There is no timeout here, so
		we may wait forever.
		"""
		self.getconnected(0.1)
		if self.connected:
			try:
				self.conn.sendto(s, self.remoteaddr)
			except socket.error:
				self.connected = False
It also does logging, but you can ignore that if you like. The design is that the methods put_ch, get_ch, and flush are called from elsewhere. Those then call the subclass-specific methods to wait for a connection and to fetch a buffer-load of bytes when they run out. You may want to separate out the connection; I wanted it to be invisible.

Return to “Python”