what do these abort_cmd(1324) and abort_task_set(1348) messages in the logs mean?

what do these abort_cmd(1324) and abort_task_set(1348) messages in the logs mean?

[Jelle de Jong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hello everybody,

What do these tgtd messages mean? Is something wrong? Why are they there?

http://paste.debian.net/105427/

Kind regards,

Jelle de Jong
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iJwEAQECAAYFAlOgF0EACgkQ1WclBW9j5HmUqgQAj4mA+i6EjyIqGXo4CoDYbc4n
vESADi+p4CGbzcBVu2AdmJnocCltDn6GlCPlMB2mtxNR8qpL6eeKLGWoXpec4eCE
Z49lG0lTcke6S6mzZqkUrOVcYVAteDprbfu4nA61OcMkfEsT4oYoG5CzfLcXH2F3
lzbigsRlx4riSEtO7E4=
=8ya9
-----END PGP SIGNATURE-----

Re: what do these abort_cmd(1324) and abort_task_set(1348) messages in the logs mean?

[Jelle de Jong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 17/06/14 12:24, Jelle de Jong wrote:
> What do these tgtd messages mean? Is something wrong? Why are they
> there?
> 
> http://paste.debian.net/105427/

Anyone that can shed some light on this, my logs are flooded with
these messages.

Is this a bug, is it fixed in a newer release?

Kind regards,

Jelle de Jong
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iJwEAQECAAYFAlOtTIkACgkQ1WclBW9j5HmfhwP9H0bk8B2mLZ7iq1FnlnotlKGo
tdU6YPJqG2VHFg7g9l9B2FgdOScZkNdULxQdbLT5foFJ+VBwKQ3cb5bUYNUB3vBh
gA+wsIgmVPQNgKQm3iSzPD+WCU3BeddEKrqs35Bfto91EcaxqyYTbjDfBipMBw9/
VKzle1Ja8sS8kPvZhWU=
=6f6z
-----END PGP SIGNATURE-----

Re: what do these abort_cmd(1324) and abort_task_set(1348) messages in the logs mean?

[FUJITA Tomonori]

On Tue, 17 Jun 2014 12:24:07 +0200
Jelle de Jong <jelledejong@powercraft.nl> wrote:

> What do these tgtd messages mean? Is something wrong? Why are they there?
> 
> http://paste.debian.net/105427/

A SCSI command was aborted for some reason (e.g. the time-out
of a SCSI command due to the overloaded iscsi target).

The cricial bug about this was fixed in 1.0.48. I would recommend you
to update tgt if you use the older version.

Re: what do these abort_cmd(1324) and abort_task_set(1348) messages in the logs mean?

[Jelle de Jong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dear FUJITA,

On 27/06/14 13:57, FUJITA Tomonori wrote:
> On Tue, 17 Jun 2014 12:24:07 +0200 Jelle de Jong
> <jelledejong@powercraft.nl> wrote:
> 
>> What do these tgtd messages mean? Is something wrong? Why are
>> they there?
>> 
>> http://paste.debian.net/105427/
> 
> A SCSI command was aborted for some reason (e.g. the time-out of a
> SCSI command due to the overloaded iscsi target).
> 
> The cricial bug about this was fixed in 1.0.48. I would recommend
> you to update tgt if you use the older version.

I upgraded to version 1.0.48 about a week ago for further testing, I
also added a complete new network path, with new network cards and
direct connections, but I keep getting these same errors:

http://paste.debian.net/108884/

Is this a bug in tgtd? Is there some tgtd option or environment
variable that I can set to get more information.

This system have been stable for more then a year with very old
versions 1.0.17 of tgt but these versions didn't shutdown the target
very reliable when using pacemaker and corosync, making it hang (I
didn't experience this with the newer versions).

Kind regards,

Jelle de Jong



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iJwEAQECAAYFAlO9TooACgkQ1WclBW9j5HnfrAP+KNwdgBC1RNQKxQiT81ZiVLA7
p0OwND1AFyusnflzjO7qecNZ5kvTL/Z5zoRM+P9ecw9kmGFRQojPsYOof/7nqMFA
UCx5sI7Jp9XNW/C6ZYX6QM//t1d0UPCAIDwciG37a/Lpymo5xj/UkJ5JcZg1rCaB
Nsfq1SRcYHVevUIVRaw=
=L21b
-----END PGP SIGNATURE-----

Re: what do these abort_cmd(1324) and abort_task_set(1348) messages in the logs mean?

[FUJITA Tomonori]

On Wed, 09 Jul 2014 16:15:42 +0200
Jelle de Jong <jelledejong@powercraft.nl> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Dear FUJITA,
> 
> On 27/06/14 13:57, FUJITA Tomonori wrote:
>> On Tue, 17 Jun 2014 12:24:07 +0200 Jelle de Jong
>> <jelledejong@powercraft.nl> wrote:
>> 
>>> What do these tgtd messages mean? Is something wrong? Why are
>>> they there?
>>> 
>>> http://paste.debian.net/105427/
>> 
>> A SCSI command was aborted for some reason (e.g. the time-out of a
>> SCSI command due to the overloaded iscsi target).
>> 
>> The cricial bug about this was fixed in 1.0.48. I would recommend
>> you to update tgt if you use the older version.
> 
> I upgraded to version 1.0.48 about a week ago for further testing, I
> also added a complete new network path, with new network cards and
> direct connections, but I keep getting these same errors:
> 
> http://paste.debian.net/108884/
> 
> Is this a bug in tgtd? Is there some tgtd option or environment
> variable that I can set to get more information.

It's not a bug. It's just a message tells that SCSI command time-out
happens for some reasons.

Re: what do these abort_cmd(1324) and abort_task_set(1348) messages in the logs mean?

[Jelle de Jong]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Dear FUJITA,

On 09/07/14 16:16, FUJITA Tomonori wrote:
> On Wed, 09 Jul 2014 16:15:42 +0200 Jelle de Jong
> <jelledejong@powercraft.nl> wrote:
> 
>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
>> 
>> Dear FUJITA,
>> 
>> On 27/06/14 13:57, FUJITA Tomonori wrote:
>>> On Tue, 17 Jun 2014 12:24:07 +0200 Jelle de Jong 
>>> <jelledejong@powercraft.nl> wrote:
>>> 
>>>> What do these tgtd messages mean? Is something wrong? Why
>>>> are they there?
>>>> 
>>>> http://paste.debian.net/105427/
>>> 
>>> A SCSI command was aborted for some reason (e.g. the time-out
>>> of a SCSI command due to the overloaded iscsi target).
>>> 
>>> The cricial bug about this was fixed in 1.0.48. I would
>>> recommend you to update tgt if you use the older version.
>> 
>> I upgraded to version 1.0.48 about a week ago for further
>> testing, I also added a complete new network path, with new
>> network cards and direct connections, but I keep getting these
>> same errors:
>> 
>> http://paste.debian.net/108884/
>> 
>> Is this a bug in tgtd? Is there some tgtd option or environment 
>> variable that I can set to get more information.
> 
> It's not a bug. It's just a message tells that SCSI command
> time-out happens for some reasons.

Can you help me with some more information to solve the problems?

Is this SCSI command time-out something that happened on the target
server with the storage, my storage is having a hard time with high
latencies, but I am not having any hardware warnings/errors. Or is
this related to the iSCSI client servers as well?

I am using open-iscsi as client:

http://paste.debian.net/108898/

These are my node records and settings, is there a setting I should
try to change or optimise for tgtd?

Kind regards,

Jelle de Jong
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)

iJwEAQECAAYFAlO9UqoACgkQ1WclBW9j5HlxpAP/cHFuiSZX9YXA/lDy9CuLY0EH
dI1Qa6apz0V7O2qD9BD3KUajTUx6tx4iDfm1WeBs2DeQxFDeNj7f/tBAY4eIpRnL
91IYALKuhCcur1aLPcsYQPE+E3cgt0MaeAzFUdTH36786G8CuJSETg5gN0hJIiJD
b39ofAaNb63hNF3pNbY=
=JxXb
-----END PGP SIGNATURE-----

Re: what do these abort_cmd(1324) and abort_task_set(1348) messages in the logs mean?

[FUJITA Tomonori]

On Wed, 09 Jul 2014 16:33:18 +0200
Jelle de Jong <jelledejong@powercraft.nl> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Dear FUJITA,
> 
> On 09/07/14 16:16, FUJITA Tomonori wrote:
>> On Wed, 09 Jul 2014 16:15:42 +0200 Jelle de Jong
>> <jelledejong@powercraft.nl> wrote:
>> 
>>> -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
>>> 
>>> Dear FUJITA,
>>> 
>>> On 27/06/14 13:57, FUJITA Tomonori wrote:
>>>> On Tue, 17 Jun 2014 12:24:07 +0200 Jelle de Jong 
>>>> <jelledejong@powercraft.nl> wrote:
>>>> 
>>>>> What do these tgtd messages mean? Is something wrong? Why
>>>>> are they there?
>>>>> 
>>>>> http://paste.debian.net/105427/
>>>> 
>>>> A SCSI command was aborted for some reason (e.g. the time-out
>>>> of a SCSI command due to the overloaded iscsi target).
>>>> 
>>>> The cricial bug about this was fixed in 1.0.48. I would
>>>> recommend you to update tgt if you use the older version.
>>> 
>>> I upgraded to version 1.0.48 about a week ago for further
>>> testing, I also added a complete new network path, with new
>>> network cards and direct connections, but I keep getting these
>>> same errors:
>>> 
>>> http://paste.debian.net/108884/
>>> 
>>> Is this a bug in tgtd? Is there some tgtd option or environment 
>>> variable that I can set to get more information.
>> 
>> It's not a bug. It's just a message tells that SCSI command
>> time-out happens for some reasons.
> 
> Can you help me with some more information to solve the problems?
> 
> Is this SCSI command time-out something that happened on the target
> server with the storage, my storage is having a hard time with high
> latencies, but I am not having any hardware warnings/errors. Or is
> this related to the iSCSI client servers as well?

Without any hardware errors, a target could takes long time to execute
SCSI commands (e.g., simply overloaded). It depends on your
environment.

You could set a longer value for SCSI command time-out on the client
side.