Re: Socket programming

Re: Socket programming

[Ardhan Madras]

Dear Canaan,

Thanks for your reply. It's a great clue :)
btw, i have one more question... 

In the receiver, do i need to allocate for example
bar->name member manually? or it's done by recv() ?


--- canaan.kao@gmail.com wrote:

From:	Canaan Kao <canaan.kao@gmail.com>
To:	ajhwb@knac.com
Cc:	linux-c-programming@vger.kernel.org
Subject: Re: Socket programming
Date:	Sun, 22 Mar 2009 21:00:14 +0800

Dear Ardhan,

You can add a mySize data member in struct foo.

struct foo
 {
  unsigned mySize;
  char *name;
  char *pix;
  int id;
 };


 struct foo *bar = malloc (sizeof(struct foo));

 if(NULL!=bar)bar->mySize=sizeof(struct foo);

 /* fill the structure, then send */

Finally, your receiver will get the size of foo in the beginning of the stream.

Best regards,
Canaan

On Sun, Mar 22, 2009 at 8:42 PM, Ardhan Madras <ajhwb@knac.com> wrote:
> Hi All,
>
> I was written small network utility in Linux 2.6, glibc 2.7.
> I have been using send() and recv() system call in SOCK_STREAM
> to send or receive fixed size data. for example, i write my data
> structure like this:
>
> struct foo
> {
>  char name[16];
>  char pix[1024];
>  int id;
> } bar;
>
> If i want to send or receive data i just call
>
> send (socket, &bar, sizeof(struct foo), 0);
>
> or to receive:
> recv (socket, &bar, sizeof(struct foo), 0);
>
> My problem is how to receive dynamic sized data? for
> example this structure:
>
> struct foo
> {
>  char *name;
>  char *pix;
>  int id;
> }
>
> struct foo *bar = malloc (sizeof(struct foo));
> /* fill the structure, then send */
>
> How to receive the data? since there are no way
> to the receiver to know data sizes?
>
> Thanks for your help.
>
>
> _____________________________________________________________
> Listen to KNAC, Hit the Home page and Tune In Live! ---> http://www.knac.com
> --
> To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html




_____________________________________________________________
Listen to KNAC, Hit the Home page and Tune In Live! ---> http://www.knac.com

Re: Socket programming

[Volker Kokula]

Hi Ardhan,


you're going to write an enterly unportable and insecure program. If you 
want just send a structure from one host to another look for a library 
which does the packing and unpacking over the network for you.

For example json-c [1] or csoap [2]


Volker



[1] http://oss.metaparadigm.com/json-c/
[2] http://sourceforge.net/projects/csoap/

Re: Socket programming

[Canaan Kao]

Dear Ardhan,

We need to allocate the bar->name manually in general.

About the related socket programming issues, I recommend you one book.
UNIX Network Programming, Volume 1, Second Edition: Networking APIs:
Sockets and XTI, Prentice Hall, 1998, ISBN 0-13-490012-X.

Best regards,
Canaan

On Sun, Mar 22, 2009 at 9:31 PM, Ardhan Madras <ajhwb@knac.com> wrote:
> Dear Canaan,
>
> Thanks for your reply. It's a great clue :)
> btw, i have one more question...
>
> In the receiver, do i need to allocate for example
> bar->name member manually? or it's done by recv() ?
>
>
> --- canaan.kao@gmail.com wrote:
>
> From:   Canaan Kao <canaan.kao@gmail.com>
> To:     ajhwb@knac.com
> Cc:     linux-c-programming@vger.kernel.org
> Subject: Re: Socket programming
> Date:   Sun, 22 Mar 2009 21:00:14 +0800
>
> Dear Ardhan,
>
> You can add a mySize data member in struct foo.
>
> struct foo
>  {
>  unsigned mySize;
>  char *name;
>  char *pix;
>  int id;
>  };
>
>
>  struct foo *bar = malloc (sizeof(struct foo));
>
>  if(NULL!=bar)bar->mySize=sizeof(struct foo);
>
>  /* fill the structure, then send */
>
> Finally, your receiver will get the size of foo in the beginning of the stream.
>
> Best regards,
> Canaan
>
> On Sun, Mar 22, 2009 at 8:42 PM, Ardhan Madras <ajhwb@knac.com> wrote:
>> Hi All,
>>
>> I was written small network utility in Linux 2.6, glibc 2.7.
>> I have been using send() and recv() system call in SOCK_STREAM
>> to send or receive fixed size data. for example, i write my data
>> structure like this:
>>
>> struct foo
>> {
>>  char name[16];
>>  char pix[1024];
>>  int id;
>> } bar;
>>
>> If i want to send or receive data i just call
>>
>> send (socket, &bar, sizeof(struct foo), 0);
>>
>> or to receive:
>> recv (socket, &bar, sizeof(struct foo), 0);
>>
>> My problem is how to receive dynamic sized data? for
>> example this structure:
>>
>> struct foo
>> {
>>  char *name;
>>  char *pix;
>>  int id;
>> }
>>
>> struct foo *bar = malloc (sizeof(struct foo));
>> /* fill the structure, then send */
>>
>> How to receive the data? since there are no way
>> to the receiver to know data sizes?
>>
>> Thanks for your help.
>>
>>
>> _____________________________________________________________
>> Listen to KNAC, Hit the Home page and Tune In Live! ---> http://www.knac.com
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
>
>
> _____________________________________________________________
> Listen to KNAC, Hit the Home page and Tune In Live! ---> http://www.knac.com
>
--
To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Socket programming

[Anil Vishnoi]

Ardhan,
Book suggested by Canaan is really good one, I also recommend.

Cheers
Anil 

-----Original Message-----
From: linux-c-programming-owner@vger.kernel.org
[mailto:linux-c-programming-owner@vger.kernel.org] On Behalf Of Canaan Kao
Sent: Sunday, 22 March 2009 20:55
To: ajhwb@knac.com
Cc: linux-c-programming@vger.kernel.org
Subject: Re: Socket programming

Dear Ardhan,

We need to allocate the bar->name manually in general.

About the related socket programming issues, I recommend you one book.
UNIX Network Programming, Volume 1, Second Edition: Networking APIs:
Sockets and XTI, Prentice Hall, 1998, ISBN 0-13-490012-X.

Best regards,
Canaan

On Sun, Mar 22, 2009 at 9:31 PM, Ardhan Madras <ajhwb@knac.com> wrote:
> Dear Canaan,
>
> Thanks for your reply. It's a great clue :)
> btw, i have one more question...
>
> In the receiver, do i need to allocate for example
> bar->name member manually? or it's done by recv() ?
>
>
> --- canaan.kao@gmail.com wrote:
>
> From:   Canaan Kao <canaan.kao@gmail.com>
> To:     ajhwb@knac.com
> Cc:     linux-c-programming@vger.kernel.org
> Subject: Re: Socket programming
> Date:   Sun, 22 Mar 2009 21:00:14 +0800
>
> Dear Ardhan,
>
> You can add a mySize data member in struct foo.
>
> struct foo
>  {
>  unsigned mySize;
>  char *name;
>  char *pix;
>  int id;
>  };
>
>
>  struct foo *bar = malloc (sizeof(struct foo));
>
>  if(NULL!=bar)bar->mySize=sizeof(struct foo);
>
>  /* fill the structure, then send */
>
> Finally, your receiver will get the size of foo in the beginning of the
stream.
>
> Best regards,
> Canaan
>
> On Sun, Mar 22, 2009 at 8:42 PM, Ardhan Madras <ajhwb@knac.com> wrote:
>> Hi All,
>>
>> I was written small network utility in Linux 2.6, glibc 2.7.
>> I have been using send() and recv() system call in SOCK_STREAM
>> to send or receive fixed size data. for example, i write my data
>> structure like this:
>>
>> struct foo
>> {
>>  char name[16];
>>  char pix[1024];
>>  int id;
>> } bar;
>>
>> If i want to send or receive data i just call
>>
>> send (socket, &bar, sizeof(struct foo), 0);
>>
>> or to receive:
>> recv (socket, &bar, sizeof(struct foo), 0);
>>
>> My problem is how to receive dynamic sized data? for
>> example this structure:
>>
>> struct foo
>> {
>>  char *name;
>>  char *pix;
>>  int id;
>> }
>>
>> struct foo *bar = malloc (sizeof(struct foo));
>> /* fill the structure, then send */
>>
>> How to receive the data? since there are no way
>> to the receiver to know data sizes?
>>
>> Thanks for your help.
>>
>>
>> _____________________________________________________________
>> Listen to KNAC, Hit the Home page and Tune In Live! --->
http://www.knac.com
>> --
>> To unsubscribe from this list: send the line "unsubscribe
linux-c-programming" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> --
> To unsubscribe from this list: send the line "unsubscribe
linux-c-programming" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
>
>
> _____________________________________________________________
> Listen to KNAC, Hit the Home page and Tune In Live! --->
http://www.knac.com
>
--
To unsubscribe from this list: send the line "unsubscribe
linux-c-programming" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Socket programming

[Ardhan Madras]

In the glibc man about send() and recv(), it says:

" The only difference between send() and write() is the presence of flags.  
  With zero  flags parameter,  send()  is equivalent to write(). ..."

--- ajhwb


--- glynn@gclements.plus.com wrote:

Apart from the comments which have already been made:

There's no reason to use send() and recv() if the flags argument is
zero. You may as well use read() and write(). Or convert the
descriptor to a FILE* with fdopen() and use fread() and fwrite().

-- 
Glynn Clements <glynn@gclements.plus.com>




_____________________________________________________________
Listen to KNAC, Hit the Home page and Tune In Live! ---> http://www.knac.com

Re: Socket programming

[Ardhan Madras]

Thanks to Canaan Kao and Volker Kokula for sharing their knowledge.

Ardhan Madras.

_____________________________________________________________
Listen to KNAC, Hit the Home page and Tune In Live! ---> http://www.knac.com

Re: Socket programming

[Ardhan Madras]

Hi Volker,

Thanks for let me know. actually, there are no security 
concerns in my program. What do you means with 'unportable?' 
do you means CPU bit type? like integer size?, i want to 
learn socket programming, so i must understand it from scratch, 
not simply by using a library wrapper.


--- kokula@informatik.uni-wuerzburg.de wrote:

From: Volker Kokula <kokula@informatik.uni-wuerzburg.de>
To: ajhwb@knac.com
CC: linux-c-programming@vger.kernel.org
Subject: Re: Socket programming
Date: Sun, 22 Mar 2009 14:41:22 +0100

Hi Ardhan,


you're going to write an enterly unportable and insecure program. If you 
want just send a structure from one host to another look for a library 
which does the packing and unpacking over the network for you.

For example json-c [1] or csoap [2]


Volker



[1] http://oss.metaparadigm.com/json-c/
[2] http://sourceforge.net/projects/csoap/




_____________________________________________________________
Listen to KNAC, Hit the Home page and Tune In Live! ---> http://www.knac.com

Re: Socket programming

[Volker Kokula]

Hi,


yes that what i meant with unportable :-) I would recommend you to read
,,Beej's Guide to Network Programming'' a imo really good ebook about 
network programming using sockets


Volker



[1] http://beej.us/guide/bgnet/

Socket programming

[Ardhan Madras]

Hi All,

I was written small network utility in Linux 2.6, glibc 2.7.
I have been using send() and recv() system call in SOCK_STREAM
to send or receive fixed size data. for example, i write my data 
structure like this:

struct foo
{
  char name[16];
  char pix[1024];
  int id;
} bar;

If i want to send or receive data i just call

send (socket, &bar, sizeof(struct foo), 0);

or to receive:
recv (socket, &bar, sizeof(struct foo), 0);

My problem is how to receive dynamic sized data? for 
example this structure:

struct foo
{
  char *name;
  char *pix;
  int id;
}

struct foo *bar = malloc (sizeof(struct foo));
/* fill the structure, then send */

How to receive the data? since there are no way
to the receiver to know data sizes?

Thanks for your help.


_____________________________________________________________
Listen to KNAC, Hit the Home page and Tune In Live! ---> http://www.knac.com

Re: Socket programming

[Canaan Kao]

Dear Ardhan,

You can add a mySize data member in struct foo.

struct foo
 {
  unsigned mySize;
  char *name;
  char *pix;
  int id;
 };


 struct foo *bar = malloc (sizeof(struct foo));

 if(NULL!=bar)bar->mySize=sizeof(struct foo);

 /* fill the structure, then send */

Finally, your receiver will get the size of foo in the beginning of the stream.

Best regards,
Canaan

On Sun, Mar 22, 2009 at 8:42 PM, Ardhan Madras <ajhwb@knac.com> wrote:
> Hi All,
>
> I was written small network utility in Linux 2.6, glibc 2.7.
> I have been using send() and recv() system call in SOCK_STREAM
> to send or receive fixed size data. for example, i write my data
> structure like this:
>
> struct foo
> {
>  char name[16];
>  char pix[1024];
>  int id;
> } bar;
>
> If i want to send or receive data i just call
>
> send (socket, &bar, sizeof(struct foo), 0);
>
> or to receive:
> recv (socket, &bar, sizeof(struct foo), 0);
>
> My problem is how to receive dynamic sized data? for
> example this structure:
>
> struct foo
> {
>  char *name;
>  char *pix;
>  int id;
> }
>
> struct foo *bar = malloc (sizeof(struct foo));
> /* fill the structure, then send */
>
> How to receive the data? since there are no way
> to the receiver to know data sizes?
>
> Thanks for your help.
>
>
> _____________________________________________________________
> Listen to KNAC, Hit the Home page and Tune In Live! ---> http://www.knac.com
> --
> To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Socket programming

[Jon Mayo]

On Sun, Mar 22, 2009 at 5:42 AM, Ardhan Madras <ajhwb@knac.com> wrote:
> Hi All,
>
> I was written small network utility in Linux 2.6, glibc 2.7.
> I have been using send() and recv() system call in SOCK_STREAM
> to send or receive fixed size data. for example, i write my data
> structure like this:
>
> struct foo
> {
>  char name[16];
>  char pix[1024];
>  int id;
> } bar;
>
(cut)

When I deal with serializing/deserializing data over a stream socket I
prefer to define a few common "atoms" and build a complete system from
that. For example I see you have char and int in your struct.

/* p : pointer to one or more ints
 * count : number of ints
 */
int recv_int (int s, int *p, unsigned count)
{
 int total, res;

 /* a loop that ensures the entire data is sent - if you wish to have
a program that handles multiple sockets using select() you need to do
this differently, like with one big read into a large buffer */
 for(total=0; total < (count * sizeof *p); total+=res) {
   res=recv (s, (char*)p+total, (count * sizeof *p)-total, 0);
   if (res=<0) return 0; /* closed or there was an error */
 }

 /* you could byte swap each element after receiving the data */

 return total;
}

/* p : pointer to one or more ints
 * count : number of ints
 */
int send_int (int s, int *p, unsigned count)
{
 int total, res;

 /* you could byte swap each element before sending */


 /* a loop that ensures the entire data is sent - if you wish to have
a program that handles multiple sockets using select() you need to do
this differently, like copy encoded data into a large output buffer
first */
 for(total=0; total < (count * sizeof *p); total+=res) {
   res=recv (s, (char*)p+total, (count * sizeof *p)-total, 0);
   if (res=<0) return 0; /* closed or there was an error */
 }

 return total;
}

/* for completeness - not significantly different from recv()/send() */
int recv_char(int s, char *p, unsigned count) { return recv (s, p,
count * sizeof *p, 0); }
int send_char(int s, char *p, unsigned count) { return send (s, p,
count * sizeof *p, 0); }



Then from those atoms you build up calls for your particular data structure:

int recv_foo(int s, struct foo *f)
{
 int res;
 /* if you want name and pix to be variable size you could read an
int for the length, then malloc() space for it, then read the bytes in
*/
 res=recv_char(s, f->name, sizeof f->name);
 if (res<=0) return res;
 res=recv_char(s, f->pix, sizeof f->name);
 if (res<=0) return res;
 res=recv_int(s, &f->id, 1);
 if (res<=0) return res;
 return 1;
}


I didn't actually try or compile any of it, just typed up a rough idea
of how you can do it.

A popular alternatives to your hand made binary protocol include:
* Just use an text based protocol. That's how HTTP, SMTP, NNTP, IRC, etc work.
* BEEP Core protocol - text based with easy ways for handling binary data
* ASN.1 - a standardized encoding and description format for binary
data (mainly for protocols). asn1c is a utility to generate C
serialize stubs from ASN.1 descriptions.
* SunRPC and XDR - part of 'rpcgen' utility for defining standard
SunRPC encode/decode.
* XML rpc - a standardized form for using XML as a protocol.

If you must have binary I would recommend you go with ASN.1 or SunRPC
over writing your own thing. But overall I would recommend one of the
text based options. The stuff to learn about socket programming will
still be there even if you use someone else to serialize the data for
your. Encoding data is a completely different skill that is separate
from socket programming.

RE: Socket programming

[Anil Vishnoi]

Hi,
	I think solution suggested by Jon is the complete one, because if I understand correctly your intent is to replicate the dynamic data which is pointed by the pointer member of the struct foo too on the other side of socket. Solution given by Canaan will possibly solve the half of the problem, but by that way you will not be able to send the content pointed by *name & *pix in the struct foo. So you need to send those data as well across the socket (which jon is doing) and need to allocate the memory equal to the string pointed by the *name & *pix and need to cast accordingly.

Cheers 
Anil 

-----Original Message-----
From: linux-c-programming-owner@vger.kernel.org [mailto:linux-c-programming-owner@vger.kernel.org] On Behalf Of Jon Mayo
Sent: Monday, 23 March 2009 00:09
To: ajhwb@knac.com; linux-c-programming@vger.kernel.org
Subject: Re: Socket programming

On Sun, Mar 22, 2009 at 5:42 AM, Ardhan Madras <ajhwb@knac.com> wrote:
> Hi All,
>
> I was written small network utility in Linux 2.6, glibc 2.7.
> I have been using send() and recv() system call in SOCK_STREAM
> to send or receive fixed size data. for example, i write my data
> structure like this:
>
> struct foo
> {
>  char name[16];
>  char pix[1024];
>  int id;
> } bar;
>
(cut)

When I deal with serializing/deserializing data over a stream socket I
prefer to define a few common "atoms" and build a complete system from
that. For example I see you have char and int in your struct.

/* p : pointer to one or more ints
 * count : number of ints
 */
int recv_int (int s, int *p, unsigned count)
{
 int total, res;

 /* a loop that ensures the entire data is sent - if you wish to have
a program that handles multiple sockets using select() you need to do
this differently, like with one big read into a large buffer */
 for(total=0; total < (count * sizeof *p); total+=res) {
   res=recv (s, (char*)p+total, (count * sizeof *p)-total, 0);
   if (res=<0) return 0; /* closed or there was an error */
 }

 /* you could byte swap each element after receiving the data */

 return total;
}

/* p : pointer to one or more ints
 * count : number of ints
 */
int send_int (int s, int *p, unsigned count)
{
 int total, res;

 /* you could byte swap each element before sending */


 /* a loop that ensures the entire data is sent - if you wish to have
a program that handles multiple sockets using select() you need to do
this differently, like copy encoded data into a large output buffer
first */
 for(total=0; total < (count * sizeof *p); total+=res) {
   res=recv (s, (char*)p+total, (count * sizeof *p)-total, 0);
   if (res=<0) return 0; /* closed or there was an error */
 }

 return total;
}

/* for completeness - not significantly different from recv()/send() */
int recv_char(int s, char *p, unsigned count) { return recv (s, p,
count * sizeof *p, 0); }
int send_char(int s, char *p, unsigned count) { return send (s, p,
count * sizeof *p, 0); }



Then from those atoms you build up calls for your particular data structure:

int recv_foo(int s, struct foo *f)
{
 int res;
 /* if you want name and pix to be variable size you could read an
int for the length, then malloc() space for it, then read the bytes in
*/
 res=recv_char(s, f->name, sizeof f->name);
 if (res<=0) return res;
 res=recv_char(s, f->pix, sizeof f->name);
 if (res<=0) return res;
 res=recv_int(s, &f->id, 1);
 if (res<=0) return res;
 return 1;
}


I didn't actually try or compile any of it, just typed up a rough idea
of how you can do it.

A popular alternatives to your hand made binary protocol include:
* Just use an text based protocol. That's how HTTP, SMTP, NNTP, IRC, etc work.
* BEEP Core protocol - text based with easy ways for handling binary data
* ASN.1 - a standardized encoding and description format for binary
data (mainly for protocols). asn1c is a utility to generate C
serialize stubs from ASN.1 descriptions.
* SunRPC and XDR - part of 'rpcgen' utility for defining standard
SunRPC encode/decode.
* XML rpc - a standardized form for using XML as a protocol.

If you must have binary I would recommend you go with ASN.1 or SunRPC
over writing your own thing. But overall I would recommend one of the
text based options. The stuff to learn about socket programming will
still be there even if you use someone else to serialize the data for
your. Encoding data is a completely different skill that is separate
from socket programming.
--
To unsubscribe from this list: send the line "unsubscribe linux-c-programming" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Socket programming

[Glynn Clements]

Ardhan Madras wrote:

> I was written small network utility in Linux 2.6, glibc 2.7.
> I have been using send() and recv() system call in SOCK_STREAM
> to send or receive fixed size data. for example, i write my data 
> structure like this:
> 
> struct foo
> {
>   char name[16];
>   char pix[1024];
>   int id;
> } bar;
> 
> If i want to send or receive data i just call
> 
> send (socket, &bar, sizeof(struct foo), 0);
> 
> or to receive:
> recv (socket, &bar, sizeof(struct foo), 0);

Apart from the comments which have already been made:

There's no reason to use send() and recv() if the flags argument is
zero. You may as well use read() and write(). Or convert the
descriptor to a FILE* with fdopen() and use fread() and fwrite().

-- 
Glynn Clements <glynn@gclements.plus.com>