[PATCH] omapdce : clear transaction if engine_open rpsend fails

[PATCH] omapdce : clear transaction if engine_open rpsend fails

[Varun B Patil]

The next time the same transaction structure is used, when the req_id's
wrap around, it should be completely cleared (memset) to avoid corruption.

Signed-off-by: Varun B Patil <varun.basavaraj.patil@gmail.com>
---
 drivers/staging/omapdce/dce.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/staging/omapdce/dce.c b/drivers/staging/omapdce/dce.c
index 1d9b75c..2dbb24e9 100644
--- a/drivers/staging/omapdce/dce.c
+++ b/drivers/staging/omapdce/dce.c
@@ -754,7 +754,7 @@ static int ioctl_engine_open(struct drm_device *dev, void *data,
 		strncpy(req.name, arg->name, sizeof(req.name));
 		ret = rpsend(priv, &arg->token, hdr(&req), sizeof(req));
 		if (ret)
-			return ret;
+			return rpabort(hdr(&req), ret);
 	}
 
 	/* then wait for reply, which is interruptible */
-- 
1.8.1.2

[PATCH] omapdce : fix bug in ioctl_codec_process

[Varun B Patil]

Do not send already free'd pointer to rpabort when rpsend fails.

Signed-off-by: Varun B Patil <varun.basavaraj.patil@gmail.com>
---
 drivers/staging/omapdce/dce.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/staging/omapdce/dce.c b/drivers/staging/omapdce/dce.c
index 2a7cbef..1d9b75c 100644
--- a/drivers/staging/omapdce/dce.c
+++ b/drivers/staging/omapdce/dce.c
@@ -1318,15 +1318,17 @@ static int ioctl_codec_process(struct drm_device *dev, void *data,
 
 		ret = rpsend(priv, &arg->token, hdr(req), ptr - (void *)req);
 rpsend_out:
-		kfree(req);
 		if (ret) {
 			/* if input buffer is already locked, unlock it now so we
 			 * don't have a leak:
 			 */
 			if (input_id)
 				codec_unlockbuf(priv, arg->codec_handle, input_id);
-			return rpabort(hdr(req), ret);
+			ret = rpabort(hdr(req), ret);
+			kfree(req);
+			return ret;
 		}
+		kfree(req);
 	}
 
 	/* then wait for reply, which is interruptible */
-- 
1.8.1.2

[PATCH] omapdce : fix transaction corruption due to wrong request id

[Varun B Patil]

ioctl_codec_process was calling get_paddr() with a wrong pointer as
argument which resulted in wrong req_id being used to get the transaction
inside get_paddr().

Signed-off-by: Varun B Patil <varun.basavaraj.patil@gmail.com>
---
 drivers/staging/omapdce/dce.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/staging/omapdce/dce.c b/drivers/staging/omapdce/dce.c
index 87a45fa..2a7cbef 100644
--- a/drivers/staging/omapdce/dce.c
+++ b/drivers/staging/omapdce/dce.c
@@ -1296,7 +1296,7 @@ static int ioctl_codec_process(struct drm_device *dev, void *data,
 
 		codec_get_quirks(priv, arg->codec_handle, &quirks);
 
-		ret = PTR_RET(get_paddr(priv, hdr(&req), &req->out_args, arg->out_args_bo));
+		ret = PTR_RET(get_paddr(priv, hdr(req), &req->out_args, arg->out_args_bo));
 		if (ret)
 			return rpabort(hdr(req), ret);
 
-- 
1.8.1.2