Re: Bus-simulation with SocketCAN

Re: Bus-simulation with SocketCAN

[Oliver Hartkopp]

Hello Sebastian,

On 09.01.2012 23:33, Sebastian Haas wrote:

> just in case someone is interested in an easy to way to simulate CAN nodes or
> to test ECUs via CAN. I want to inform you that my recently started project
> restcan [3] reached a first milestone. Look at [1] to know what restcan is.
> 
> The milestone includes the following features:
> - support of SocketCANs' RAW channels incl. Rx filters
> - support of a rudimentary JSON-based message database format


Do you know the Kayak project from Jan-Niklas Meier?

Kajak is a JAVA based frontend and library, see:

http://kayak.2codeornot2code.org/

http://kayak.2codeornot2code.org/tutorial.html

And on github: https://github.com/dschanoeh/Kayak

Implementing a restbus functionality in JAVA based on the Kajak library is
very easy - when  you are a JAVA developer ;-)

I like your approach to get a restbus simulation in C.

But checking your JSON description for CAN signals i stumbled about the
'intel' and 'motorola' tags which stand for 'little' and 'big' endian.

I wonder, if it makes sense for your project to adopt the XML-based KCD
format for CAN signal descriptions:

https://github.com/dschanoeh/Kayak/blob/ee6b459bddd1831ae99c1afcde01f315e4b454b4/Kayak-kcd/src/test/resources/can_definition_sample.kcd

Here is the XML schema definition:

https://github.com/dschanoeh/Kayak/blob/master/Kayak-kcd/src/main/resources/com/github/kayak/canio/kcd/schema/Definition.xsd

Maybe you can build your JSON description with an XSLT process from the well
defined KCD description.

There's also a dbc input converter on the run to create KCDs:
https://github.com/dschanoeh/CANBabel


> - support of receiving and decoding measurement values encoded in CAN
> messages, sending of such messages is also possible
> 
> See the examples in [5] to get an impression of using restcan.
> 
> Since my plans are big and I'm just one person with limited sparetime I ask
> everyone here to support this project with its mission to provide any easy to
> use way of accessing SocketCAN. See [4] for things I want to implement in the
> near future.
> 
> I hope Oliver is fine with "mis-"using the socketcan mailing list to announce
> this milestone of this very socketcan-related project. :-)


:-) This is not in my responsibility what is posted on SocketCAN-Users.
Btw. i put it onto the linux-can ML to continue the discussion there.

It's a good question, if we just do announcements on linux-can ML as i don't
know if the SocketCAN-Users ML will continue in the future.


> 
> [1] https://github.com/sebi2k1/restcan/wiki
> [2] http://sebi2k1.wordpress.com/2012/01/09/transmitting-and-receiving-physical-values-via-can/
> [3] https://github.com/sebi2k1/restcan
> [4] https://github.com/sebi2k1/restcan/blob/master/TODO.md
> [5] https://github.com/sebi2k1/restcan/tree/master/sample_scripts


Regards,
Oliver

Re: Bus-simulation with SocketCAN

[Sebastian Haas]

Hi

Am 10.01.2012 11:29, schrieb Oliver Hartkopp:
> Hello Sebastian,
>
> On 09.01.2012 23:33, Sebastian Haas wrote:
>
>> just in case someone is interested in an easy to way to simulate CAN nodes or
>> to test ECUs via CAN. I want to inform you that my recently started project
>> restcan [3] reached a first milestone. Look at [1] to know what restcan is.
>>
>> The milestone includes the following features:
>> - support of SocketCANs' RAW channels incl. Rx filters
>> - support of a rudimentary JSON-based message database format
>
>
> Do you know the Kayak project from Jan-Niklas Meier?
>
> Kajak is a JAVA based frontend and library, see:
>
> http://kayak.2codeornot2code.org/
>
> http://kayak.2codeornot2code.org/tutorial.html
>
> And on github: https://github.com/dschanoeh/Kayak
That looks quite interesting. The approach is different than mine, as 
the logic in restcan is at the server side and the client (not yet 
available) is an optional visualization/control component.

It is quite difficult to find active open source projects which are CAN 
related. I hope to find some help to implement the visualization for 
restcan. Too bad Kayak does the signal processing on its own.

> Implementing a restbus functionality in JAVA based on the Kajak library is
> very easy - when  you are a JAVA developer ;-)
>
> I like your approach to get a restbus simulation in C.
Well the simulation is written in Javascript, which is even easier than 
C or Java. I want it to be easy to use. The only dependency is NodeJS 
and you can write and run simulation and test cases from commandline 
within few minutes.

> But checking your JSON description for CAN signals i stumbled about the
> 'intel' and 'motorola' tags which stand for 'little' and 'big' endian.
Yeah, you are right. I will improve it.

> I wonder, if it makes sense for your project to adopt the XML-based KCD
> format for CAN signal descriptions:
I had a look at the spec, KCD looks pretty good, I think I will use it 
to replace the JSON description.

> https://github.com/dschanoeh/Kayak/blob/ee6b459bddd1831ae99c1afcde01f315e4b454b4/Kayak-kcd/src/test/resources/can_definition_sample.kcd
>
> Here is the XML schema definition:
>
> https://github.com/dschanoeh/Kayak/blob/master/Kayak-kcd/src/main/resources/com/github/kayak/canio/kcd/schema/Definition.xsd
>
> Maybe you can build your JSON description with an XSLT process from the well
> defined KCD description.
>
> There's also a dbc input converter on the run to create KCDs:
> https://github.com/dschanoeh/CANBabel
Thats create, I was afraid I need to implement the converter myself. ;-) 
I hope they will extend to parse .sym files from Peak as well.

Do you know about BUSMASTER (https://github.com/rbei-etas/busmaster). If 
it weren't Windows (R) only it would be an interesting solution.

Oliver, what must restcan provide to be useable for you in your daily 
work? Or what do you expect from such a tool?

Cheers,
  Sebastian

Re: Bus-simulation with SocketCAN

[Yegor Yefremov]

> Do you know about BUSMASTER (https://github.com/rbei-etas/busmaster). If it weren't Windows (R) only it would be an interesting solution.

As for now it runs only on Windows, but there is an effort to make it cross-platform i.e. Qt + CMake: https://github.com/rbei-etas/busmaster/issues/45

Yegor

Re: Bus-simulation with SocketCAN

[Sebastian Haas]

Am 11.01.2012 08:24, schrieb Yegor Yefremov:
>> Do you know about BUSMASTER (https://github.com/rbei-etas/busmaster). If it weren't Windows (R) only it would be an interesting solution.
>
> As for now it runs only on Windows, but there is an effort to make it cross-platform i.e. Qt + CMake: https://github.com/rbei-etas/busmaster/issues/45
I had a look at the soure code and it looks heavely coupled to MFC. This 
is not an easy job, but I wish them all the best. ;-)

Sebastian

Re: Bus-simulation with SocketCAN

[Jan-Niklas Meier]

Hi!

2012/1/10 Sebastian Haas <dev@sebastianhaas.info>:

> That looks quite interesting. The approach is different than mine, as the
> logic in restcan is at the server side and the client (not yet available) is
> an optional visualization/control component.
>
> It is quite difficult to find active open source projects which are CAN
> related. I hope to find some help to implement the visualization for
> restcan. Too bad Kayak does the signal processing on its own.

I agree that the approaches are quite different and also we are trying
to cover different use cases with our projects. With Kayak I tried to
provide a free CAN bus analysis solution. The TCP/IP abstraction is a
nice feature which enables you to do things like remote analysis and
to connect from multiple machines at once.
Kayak is not - and will probably never be - a very good tool for
restbus simulation. In principle it is possible but Kayak is not
intended for this.

>> Implementing a restbus functionality in JAVA based on the Kajak library is
>> very easy - when  you are a JAVA developer ;-)
>>
>> I like your approach to get a restbus simulation in C.
>
> Well the simulation is written in Javascript, which is even easier than C or
> Java. I want it to be easy to use. The only dependency is NodeJS and you can
> write and run simulation and test cases from commandline within few minutes.

I like the idea of using Javascript and especially NodeJS for restbus
simulation. At the weekend I will try to set it up at home and play
around a little bit :).

>> I wonder, if it makes sense for your project to adopt the XML-based KCD
>> format for CAN signal descriptions:
>
> I had a look at the spec, KCD looks pretty good, I think I will use it to
> replace the JSON description.

\o/ Yay! That's a good idea.

>> There's also a dbc input converter on the run to create KCDs:
>> https://github.com/dschanoeh/CANBabel
>
> Thats create, I was afraid I need to implement the converter myself. ;-) I
> hope they will extend to parse .sym files from Peak as well.

We mostly have .dbc files we need to convert. Support for different
formats (and possibly even conversion in both ways) is planned. Do you
use PEAK .sym-files and could you provide some example files and help
finding bugs in a .sym-Parser?

> Do you know about BUSMASTER (https://github.com/rbei-etas/busmaster). If it
> weren't Windows (R) only it would be an interesting solution.
>
> Oliver, what must restcan provide to be useable for you in your daily work?
> Or what do you expect from such a tool?

How lightweight is NodeJS? I would find this useful for quick and
dirty hacks in embedded units. Could memory (Flash and RAM) be a
problem on such devices?

Greetings,
Jan

Re: Bus-simulation with SocketCAN

[Sebastian Haas]

Hello together,

Am 11.01.2012 09:38, schrieb Jan-Niklas Meier:
> 2012/1/10 Sebastian Haas<dev@sebastianhaas.info>:
>>> Implementing a restbus functionality in JAVA based on the Kajak library is
>>> very easy - when  you are a JAVA developer ;-)
>>>
>>> I like your approach to get a restbus simulation in C.
>>
>> Well the simulation is written in Javascript, which is even easier than C or
>> Java. I want it to be easy to use. The only dependency is NodeJS and you can
>> write and run simulation and test cases from commandline within few minutes.
>
> I like the idea of using Javascript and especially NodeJS for restbus
> simulation. At the weekend I will try to set it up at home and play
> around a little bit :).
Sounds good. I hope you like it and we can work together on it.

>>> I wonder, if it makes sense for your project to adopt the XML-based KCD
>>> format for CAN signal descriptions:
>>
>> I had a look at the spec, KCD looks pretty good, I think I will use it to
>> replace the JSON description.
>
> \o/ Yay! That's a good idea.
Yes KCD looks quite complete to describe a whole CAN network include the 
communicate. But I'm not yet sure how to extend it for specific features 
like other protocols.

The basic idea was to have a NodeJS instance for each CAN node to 
simulate. Basically the API allows also to simulate many nodes within 
the same NodeJS instance. But I think it is easier for an developer to 
develop and test CAN nodes in different instances.

>>> There's also a dbc input converter on the run to create KCDs:
>>> https://github.com/dschanoeh/CANBabel
>>
>> Thats create, I was afraid I need to implement the converter myself. ;-) I
>> hope they will extend to parse .sym files from Peak as well.
>
> We mostly have .dbc files we need to convert. Support for different
> formats (and possibly even conversion in both ways) is planned. Do you
> use PEAK .sym-files and could you provide some example files and help
> finding bugs in a .sym-Parser?
No I do not have any PEAK files, but I wonder @Stephane may help us.

>> Do you know about BUSMASTER (https://github.com/rbei-etas/busmaster). If it
>> weren't Windows (R) only it would be an interesting solution.
>>
>> Oliver, what must restcan provide to be useable for you in your daily work?
>> Or what do you expect from such a tool?
>
> How lightweight is NodeJS? I would find this useful for quick and
> dirty hacks in embedded units. Could memory (Flash and RAM) be a
> problem on such devices?
This thought came also in my mind. I wonder on which type of machines 
(available RAM) do you want run it?

NodeJS just relies on STL and PThread. But I'm afraid STL is quite big 
;-) ... Heap usage looks quite good IMHO. If I let signal_tx.js running 
the Gnome System monitor reports 2.3MB memory usage. If you also take 
shared libaries into account the footprint explodes up to 64MB. But I 
wonder if it is possible to compile NodeJS/V8 against a more memory 
sensible version of the STL.

Cheers,
  Sebastian

Re: Bus-simulation with SocketCAN

[Sebastian Haas]

Hello Uwe,

would be great if could provide us example .sym files which utilize the 
whole feature set of the SYM file format.

Specifications or example codes are also very much appreciated. ;-)

Thanks for your support.

Cheers,
  Sebastian

Am 13.01.2012 22:09, schrieb Uwe Wilhelm:
> Dear Sebastian,
> if you need any help with the SYM File Format, please let us know.
>
> Best Regards
>
>   Uwe Wilhelm
>   PEAK-System
>   Sales&  Support
>
> -----Ursprüngliche Nachricht-----
> Von: linux-can-owner@vger.kernel.org
> [mailto:linux-can-owner@vger.kernel.org] Im Auftrag von Sebastian Haas
> Gesendet: Freitag, 13. Januar 2012 21:07
> An: Jan-Niklas Meier
> Cc: Oliver Hartkopp; linux-can@vger.kernel.org; s.grosjean@peak-system.com
> Betreff: Re: Bus-simulation with SocketCAN
>
> Hello together,
>
> Am 11.01.2012 09:38, schrieb Jan-Niklas Meier:
>> 2012/1/10 Sebastian Haas<dev@sebastianhaas.info>:
>>>> Implementing a restbus functionality in JAVA based on the Kajak library
> is
>>>> very easy - when  you are a JAVA developer ;-)
>>>>
>>>> I like your approach to get a restbus simulation in C.
>>>
>>> Well the simulation is written in Javascript, which is even easier than C
> or
>>> Java. I want it to be easy to use. The only dependency is NodeJS and you
> can
>>> write and run simulation and test cases from commandline within few
> minutes.
>>
>> I like the idea of using Javascript and especially NodeJS for restbus
>> simulation. At the weekend I will try to set it up at home and play
>> around a little bit :).
> Sounds good. I hope you like it and we can work together on it.
>
>>>> I wonder, if it makes sense for your project to adopt the XML-based KCD
>>>> format for CAN signal descriptions:
>>>
>>> I had a look at the spec, KCD looks pretty good, I think I will use it to
>>> replace the JSON description.
>>
>> \o/ Yay! That's a good idea.
> Yes KCD looks quite complete to describe a whole CAN network include the
> communicate. But I'm not yet sure how to extend it for specific features
> like other protocols.
>
> The basic idea was to have a NodeJS instance for each CAN node to
> simulate. Basically the API allows also to simulate many nodes within
> the same NodeJS instance. But I think it is easier for an developer to
> develop and test CAN nodes in different instances.
>
>>>> There's also a dbc input converter on the run to create KCDs:
>>>> https://github.com/dschanoeh/CANBabel
>>>
>>> Thats create, I was afraid I need to implement the converter myself. ;-)
> I
>>> hope they will extend to parse .sym files from Peak as well.
>>
>> We mostly have .dbc files we need to convert. Support for different
>> formats (and possibly even conversion in both ways) is planned. Do you
>> use PEAK .sym-files and could you provide some example files and help
>> finding bugs in a .sym-Parser?
> No I do not have any PEAK files, but I wonder @Stephane may help us.
>
>>> Do you know about BUSMASTER (https://github.com/rbei-etas/busmaster). If
> it
>>> weren't Windows (R) only it would be an interesting solution.
>>>
>>> Oliver, what must restcan provide to be useable for you in your daily
> work?
>>> Or what do you expect from such a tool?
>>
>> How lightweight is NodeJS? I would find this useful for quick and
>> dirty hacks in embedded units. Could memory (Flash and RAM) be a
>> problem on such devices?
> This thought came also in my mind. I wonder on which type of machines
> (available RAM) do you want run it?
>
> NodeJS just relies on STL and PThread. But I'm afraid STL is quite big
> ;-) ... Heap usage looks quite good IMHO. If I let signal_tx.js running
> the Gnome System monitor reports 2.3MB memory usage. If you also take
> shared libaries into account the footprint explodes up to 64MB. But I
> wonder if it is possible to compile NodeJS/V8 against a more memory
> sensible version of the STL.
>
> Cheers,
>    Sebastian
> --
> To unsubscribe from this list: send the line "unsubscribe linux-can" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> --
> PEAK-System Technik GmbH, Otto-Roehm-Strasse 69, D-64293 Darmstadt
> Geschaeftsleitung: A.Gach/U.Wilhelm, HRB 9183 Darmstadt, Ust.IdNr.:DE 202 220 078
> St.Nr.:007/241/13586 FA Darmstadt, WEE-Reg.-Nr.: DE39305391
> Tel.:+49 (0)6151 8173-20 - Fax:+49 (0)6151 8173-29
> E-mail: info@peak-system.com - http://www.peak-system.com

Re: Bus-simulation with SocketCAN

[Sebastian Haas]

Hello Jan-Niklas,

I just adopted the KCD file format (see [1]). restcans' signal parser is 
missing support for muxed signals and converting signed/double is also 
not yet implemented. This points are already in the restcan issue tracker.

The parsing has also one limitation, if a signal has more than one 
"Value" tag, the parser always uses the first one.

Had you already time to play a bit with restcan?

I'm going to write a bigger example to show how to simulate different 
nodes on multiple buses.

I'm also started to implement a JSON interface to access CAN and the 
signals via WebSockets from within a browser. So you can monitor the 
simulation on the browser and control certain nodes via a web interface. 
It shall also be possible to adopt this interface to be used by native 
application using a TCP/IP socket and a JSON parser.

Cheers,
  Sebastian

   [1] 
https://github.com/sebi2k1/restcan/blob/master/sample_scripts/signal_tx.js

Am 13.01.2012 21:06, schrieb Sebastian Haas:
> Hello together,
>
> Am 11.01.2012 09:38, schrieb Jan-Niklas Meier:
>> 2012/1/10 Sebastian Haas<dev@sebastianhaas.info>:
>>>> Implementing a restbus functionality in JAVA based on the Kajak
>>>> library is
>>>> very easy - when you are a JAVA developer ;-)
>>>>
>>>> I like your approach to get a restbus simulation in C.
>>>
>>> Well the simulation is written in Javascript, which is even easier
>>> than C or
>>> Java. I want it to be easy to use. The only dependency is NodeJS and
>>> you can
>>> write and run simulation and test cases from commandline within few
>>> minutes.
>>
>> I like the idea of using Javascript and especially NodeJS for restbus
>> simulation. At the weekend I will try to set it up at home and play
>> around a little bit :).
> Sounds good. I hope you like it and we can work together on it.
>
>>>> I wonder, if it makes sense for your project to adopt the XML-based KCD
>>>> format for CAN signal descriptions:
>>>
>>> I had a look at the spec, KCD looks pretty good, I think I will use
>>> it to
>>> replace the JSON description.
>>
>> \o/ Yay! That's a good idea.
> Yes KCD looks quite complete to describe a whole CAN network include the
> communicate. But I'm not yet sure how to extend it for specific features
> like other protocols.
>
> The basic idea was to have a NodeJS instance for each CAN node to
> simulate. Basically the API allows also to simulate many nodes within
> the same NodeJS instance. But I think it is easier for an developer to
> develop and test CAN nodes in different instances.
>
>>>> There's also a dbc input converter on the run to create KCDs:
>>>> https://github.com/dschanoeh/CANBabel
>>>
>>> Thats create, I was afraid I need to implement the converter myself.
>>> ;-) I
>>> hope they will extend to parse .sym files from Peak as well.
>>
>> We mostly have .dbc files we need to convert. Support for different
>> formats (and possibly even conversion in both ways) is planned. Do you
>> use PEAK .sym-files and could you provide some example files and help
>> finding bugs in a .sym-Parser?
> No I do not have any PEAK files, but I wonder @Stephane may help us.
>
>>> Do you know about BUSMASTER (https://github.com/rbei-etas/busmaster).
>>> If it
>>> weren't Windows (R) only it would be an interesting solution.
>>>
>>> Oliver, what must restcan provide to be useable for you in your daily
>>> work?
>>> Or what do you expect from such a tool?
>>
>> How lightweight is NodeJS? I would find this useful for quick and
>> dirty hacks in embedded units. Could memory (Flash and RAM) be a
>> problem on such devices?
> This thought came also in my mind. I wonder on which type of machines
> (available RAM) do you want run it?
>
> NodeJS just relies on STL and PThread. But I'm afraid STL is quite big
> ;-) ... Heap usage looks quite good IMHO. If I let signal_tx.js running
> the Gnome System monitor reports 2.3MB memory usage. If you also take
> shared libaries into account the footprint explodes up to 64MB. But I
> wonder if it is possible to compile NodeJS/V8 against a more memory
> sensible version of the STL.
>
> Cheers,
> Sebastian
> --
> To unsubscribe from this list: send the line "unsubscribe linux-can" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Bus-simulation with SocketCAN

[Jan-Niklas Meier]

Hi Sebastian, 

Am 15.01.2012 um 10:13 schrieb Sebastian Haas:

> Hello Jan-Niklas,
> 
> I just adopted the KCD file format (see [1]). restcans' signal parser is missing support for muxed signals and converting signed/double is also not yet implemented. This points are already in the restcan issue tracker.
> 
> The parsing has also one limitation, if a signal has more than one "Value" tag, the parser always uses the first one.
> 
> Had you already time to play a bit with rest can?

It took some time for me to set up Node.js and get familiar with the module and add-on stuff. I built the can module successfully and could run the simple.js example which is quite neat.
I was not able to compile and install the can_signals module. Please check the issue I opened and the commits on my fork.

> I'm going to write a bigger example to show how to simulate different nodes on multiple buses.

That would be nice.

Today I started refactoring CANBabel to be able to add support for new file types. Hopefully I did not break anything but I will have to check this at work tomorrow :).

Greetings,
Jan-Niklas