Re: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().

Re: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().

[tadeusz.struk]

From: 
Date: Sun, 16 Jan 2011 16:41:11 +0000
Subject: RE: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().

Hi Jesper,
Thanks, but I think there is still a problem here. You don't want to kfree req_data
when the kmalloc failed. I think it should look as follows.  
Are you ok with this?

On Mon, 7 Feb 2011, Herbert Xu wrote:

> On Sun, Feb 06, 2011 at 09:34:33PM +0100, Jesper Juhl wrote:
> > On Mon, 7 Feb 2011, Herbert Xu wrote:
> > 
> > > On Sun, Feb 06, 2011 at 08:43:22PM +0100, Jesper Juhl wrote:
> > > > 
> > > > Herbert: If Tadeusz agrees, could you please replace the patch
> > > > you merged 
> > > > with the one above?
> > > 
> > > Please send an incremental patch.
> > > 
> > Sure thing. What would you like it based on exactly?
> 
> The current cryptodev-2.6 tree should do.
> 
Here goes.

Fix up previous patch that attempted to fix a mem leak in 
rfc4106_set_hash_subkey. The previous patch was flawed in that the 'goto 
out' would still leak.

---
 arch/x86/crypto/aesni-intel_glue.c |    7 +++----
 1 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c
index e013552..c9f0227 100644
--- a/arch/x86/crypto/aesni-intel_glue.c
+++ b/arch/x86/crypto/aesni-intel_glue.c
@@ -874,11 +874,11 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
 
 	ret = crypto_ablkcipher_setkey(ctr_tfm, key, key_len);
 	if (ret)
-		goto out;
+		goto out_free_ablkcipher;
 
 	req = ablkcipher_request_alloc(ctr_tfm, GFP_KERNEL);
 	if (!req) {
-		ret = -EINVAL;
+		ret = -ENOMEM;
 		goto out_free_ablkcipher;
 	}
 
@@ -911,12 +911,11 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
 		if (!ret)
 			ret = req_data->result.err;
 	}
+	kfree(req_data);
 out_free_request:
 	ablkcipher_request_free(req);
-	kfree(req_data);
 out_free_ablkcipher:
 	crypto_free_ablkcipher(ctr_tfm);
-out:
 	return ret;
 }
 
-- 
1.7.4

--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

Re: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().

[Jesper Juhl]

On Mon, 7 Feb 2011, tadeusz.struk@intel.com wrote:

> From: 
> Date: Sun, 16 Jan 2011 16:41:11 +0000
> Subject: RE: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().
> 
> Hi Jesper,
> Thanks, but I think there is still a problem here. You don't want to kfree req_data
> when the kmalloc failed. I think it should look as follows.  
> Are you ok with this?
> 
Fine by me.

I was aware of the kfree(NULL) thing, but desided to leave it as is for 
two reasons - 1) kfree(NULL) is harmless and this is an error path, so the 
extra function call doesn't matter much. 2) I wanted to preserve 
deallocations in the reverse order of the allocations. But sure, moving 
that kfree is a tiny optimization of the error path, so I'm fine with it.


-- 
Jesper Juhl <jj@chaosbits.net>            http://www.chaosbits.net/
Plain text mails only, please.
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html

Re: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().

[Pavel Machek]

On Mon 2011-02-07 19:24:43, Jesper Juhl wrote:
> On Mon, 7 Feb 2011, tadeusz.struk@intel.com wrote:
> 
> > From: 
> > Date: Sun, 16 Jan 2011 16:41:11 +0000
> > Subject: RE: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().
> > 
> > Hi Jesper,
> > Thanks, but I think there is still a problem here. You don't want to kfree req_data
> > when the kmalloc failed. I think it should look as follows.  
> > Are you ok with this?
> > 
> Fine by me.
> 
> I was aware of the kfree(NULL) thing, but desided to leave it as is for 
> two reasons - 1) kfree(NULL) is harmless and this is an error path, so the 
> extra function call doesn't matter much. 2) I wanted to preserve 
> deallocations in the reverse order of the allocations. But sure, moving 
> that kfree is a tiny optimization of the error path, so I'm fine with it.

I don't think such optimalization is worth doing... original code is
as good but shorter and less complex...

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

RE: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().

[Paoloni, Gabriele]

Well anyway I think that the return value of "ablkcipher_request_alloc(ctr_tfm, GFP_KERNEL)" has to be changed from -EINVAL to -ENOMEM in case of failure. That is why would stay on the patch that Tadeusz proposed. Otherwise Juhl should send another one....

> -----Original Message-----
> From: Pavel Machek [mailto:pavel@ucw.cz]
> Sent: Friday, February 11, 2011 2:27 PM
> To: Jesper Juhl
> Cc: Struk, Tadeusz; herbert@gondor.hengli.com.au; linux-
> kernel@vger.kernel.org; linux-crypto@vger.kernel.org; O Mahony, Aidan;
> Paoloni, Gabriele; Hoban, Adrian
> Subject: Re: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in
> rfc4106_set_hash_subkey().
> 
> On Mon 2011-02-07 19:24:43, Jesper Juhl wrote:
> > On Mon, 7 Feb 2011, tadeusz.struk@intel.com wrote:
> >
> > > From:
> > > Date: Sun, 16 Jan 2011 16:41:11 +0000
> > > Subject: RE: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in
> rfc4106_set_hash_subkey().
> > >
> > > Hi Jesper,
> > > Thanks, but I think there is still a problem here. You don't want
> to kfree req_data
> > > when the kmalloc failed. I think it should look as follows.
> > > Are you ok with this?
> > >
> > Fine by me.
> >
> > I was aware of the kfree(NULL) thing, but desided to leave it as is
> for
> > two reasons - 1) kfree(NULL) is harmless and this is an error path,
> so the
> > extra function call doesn't matter much. 2) I wanted to preserve
> > deallocations in the reverse order of the allocations. But sure,
> moving
> > that kfree is a tiny optimization of the error path, so I'm fine with
> it.
> 
> I don't think such optimalization is worth doing... original code is
> as good but shorter and less complex...
> 
> --
> (english) http://www.livejournal.com/~pavelmachek
> (cesky, pictures)
> http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

RE: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().

[Jesper Juhl]

On Fri, 11 Feb 2011, Paoloni, Gabriele wrote:

> Well anyway I think that the return value of "ablkcipher_request_alloc(ctr_tfm, GFP_KERNEL)" has to be changed from -EINVAL to -ENOMEM in case of failure. That is why would stay on the patch that Tadeusz proposed. Otherwise Juhl should send another one....
> 
I'll take a look again later this evening when I get home from work.


-- 
Jesper Juhl <jj@chaosbits.net>            http://www.chaosbits.net/
Plain text mails only, please.
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html

RE: [PATCH](updated) rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey()..

[Jesper Juhl]

On Fri, 11 Feb 2011, Jesper Juhl wrote:

> On Fri, 11 Feb 2011, Paoloni, Gabriele wrote:
> 
> > Well anyway I think that the return value of "ablkcipher_request_alloc(ctr_tfm, GFP_KERNEL)" has to be changed from -EINVAL to -ENOMEM in case of failure. That is why would stay on the patch that Tadeusz proposed. Otherwise Juhl should send another one....
> > 
> I'll take a look again later this evening when I get home from work.
> 
Hopefully this takes care of all complaints and can actually get merged so 
we can get the leak fixed (patch is against current cryptodev-2.6 tree).


Fix up previous patch that failed to properly fix mem leak in 
rfc4106_set_hash_subkey(). This add-on patch; fixes the leak. moves 
kfree() out of the error path, returns -ENOMEM rather than -EINVAL when 
ablkcipher_request_alloc() fails.

Signed-off-by: Jesper Juhl <jj@chaosbits.net>
---
 aesni-intel_glue.c |   13 +++++--------
 1 file changed, 5 insertions(+), 8 deletions(-)

diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c
index e013552..502b76d 100644
--- a/arch/x86/crypto/aesni-intel_glue.c
+++ b/arch/x86/crypto/aesni-intel_glue.c
@@ -876,17 +876,15 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
 	if (ret)
 		goto out;
 
+	ret = -ENOMEM;
 	req = ablkcipher_request_alloc(ctr_tfm, GFP_KERNEL);
-	if (!req) {
-		ret = -EINVAL;
+	if (!req)
 		goto out_free_ablkcipher;
-	}
 
 	req_data = kmalloc(sizeof(*req_data), GFP_KERNEL);
-	if (!req_data) {
-		ret = -ENOMEM;
+	if (!req_data)
 		goto out_free_request;
-	}
+
 	memset(req_data->iv, 0, sizeof(req_data->iv));
 
 	/* Clear the data in the hash sub key container to zero.*/
@@ -911,12 +909,11 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
 		if (!ret)
 			ret = req_data->result.err;
 	}
+	kfree(req_data);
 out_free_request:
 	ablkcipher_request_free(req);
-	kfree(req_data);
 out_free_ablkcipher:
 	crypto_free_ablkcipher(ctr_tfm);
-out:
 	return ret;
 }
 


-- 
Jesper Juhl <jj@chaosbits.net>            http://www.chaosbits.net/
Plain text mails only, please.
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html

RE: [PATCH](updated) rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey()..

[Jesper Juhl]

On Fri, 11 Feb 2011, Jesper Juhl wrote:

> On Fri, 11 Feb 2011, Jesper Juhl wrote:
> 
> > On Fri, 11 Feb 2011, Paoloni, Gabriele wrote:
> > 
> > > Well anyway I think that the return value of "ablkcipher_request_alloc(ctr_tfm, GFP_KERNEL)" has to be changed from -EINVAL to -ENOMEM in case of failure. That is why would stay on the patch that Tadeusz proposed. Otherwise Juhl should send another one....
> > > 
> > I'll take a look again later this evening when I get home from work.
> > 
> Hopefully this takes care of all complaints and can actually get merged so 
> we can get the leak fixed (patch is against current cryptodev-2.6 tree).
> 
> 
> Fix up previous patch that failed to properly fix mem leak in 
> rfc4106_set_hash_subkey(). This add-on patch; fixes the leak. moves 
> kfree() out of the error path, returns -ENOMEM rather than -EINVAL when 
> ablkcipher_request_alloc() fails.
> 

And of course I just had to screw up and send the wrong diff. The one I 
sent does not compile and was a temporary file I imported into the mail by 
accident :-(
Below is the real patch - changelog above still applies - sorry about 
that.

Signed-off-by: Jesper Juhl <jj@chaosbits.net>
---
 aesni-intel_glue.c |   15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c
index e013552..e0e6340 100644
--- a/arch/x86/crypto/aesni-intel_glue.c
+++ b/arch/x86/crypto/aesni-intel_glue.c
@@ -874,19 +874,17 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
 
 	ret = crypto_ablkcipher_setkey(ctr_tfm, key, key_len);
 	if (ret)
-		goto out;
+		goto out_free_ablkcipher;
 
+	ret = -ENOMEM;
 	req = ablkcipher_request_alloc(ctr_tfm, GFP_KERNEL);
-	if (!req) {
-		ret = -EINVAL;
+	if (!req)
 		goto out_free_ablkcipher;
-	}
 
 	req_data = kmalloc(sizeof(*req_data), GFP_KERNEL);
-	if (!req_data) {
-		ret = -ENOMEM;
+	if (!req_data)
 		goto out_free_request;
-	}
+
 	memset(req_data->iv, 0, sizeof(req_data->iv));
 
 	/* Clear the data in the hash sub key container to zero.*/
@@ -911,12 +909,11 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
 		if (!ret)
 			ret = req_data->result.err;
 	}
+	kfree(req_data);
 out_free_request:
 	ablkcipher_request_free(req);
-	kfree(req_data);
 out_free_ablkcipher:
 	crypto_free_ablkcipher(ctr_tfm);
-out:
 	return ret;
 }
 
 
-- 
Jesper Juhl <jj@chaosbits.net>            http://www.chaosbits.net/
Plain text mails only, please.
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html

Re: [PATCH](updated) rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey()..

[Herbert Xu]

On Fri, Feb 11, 2011 at 10:14:44PM +0100, Jesper Juhl wrote:
> 
> And of course I just had to screw up and send the wrong diff. The one I 
> sent does not compile and was a temporary file I imported into the mail by 
> accident :-(
> Below is the real patch - changelog above still applies - sorry about 
> that.
> 
> Signed-off-by: Jesper Juhl <jj@chaosbits.net>

Patch applied.  Thanks!
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().

[Herbert Xu]

On Mon, Feb 07, 2011 at 05:32:59PM +0000, tadeusz.struk@intel.com wrote:
>
> Here goes.
> 
> Fix up previous patch that attempted to fix a mem leak in 
> rfc4106_set_hash_subkey. The previous patch was flawed in that the 'goto 
> out' would still leak.

Sign-off?
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().

[Jesper Juhl]

On Tue, 8 Feb 2011, Herbert Xu wrote:

> On Mon, Feb 07, 2011 at 05:32:59PM +0000, tadeusz.struk@intel.com wrote:
> >
> > Here goes.
> > 
> > Fix up previous patch that attempted to fix a mem leak in 
> > rfc4106_set_hash_subkey. The previous patch was flawed in that the 'goto 
> > out' would still leak.
> 
> Sign-off?
> 

Since it's identical to the patch I posted except for the moving of the 
kfree() which (IMHO) is a minor detail, I hope you can use my sign-off so 
we can move along and get the patch merged..

Signed-off-by: Jesper Juhl <jj@chaosbits.net>


-- 
Jesper Juhl <jj@chaosbits.net>            http://www.chaosbits.net/
Plain text mails only, please.
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html

Re: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().

[tadeusz.struk]

From: Tadeusz Struk <tadeusz.struk@intel.com>
Date: Sun, 16 Jan 2011 16:41:11 +0000
Subject: RE: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().

Hi Jesper,
Thanks, but I think there is still a problem here. You don't want to kfree req_data
when the kmalloc failed. I think it should look as follows.  
Are you ok with this?

On Mon, 7 Feb 2011, Herbert Xu wrote:

> On Sun, Feb 06, 2011 at 09:34:33PM +0100, Jesper Juhl wrote:
> > On Mon, 7 Feb 2011, Herbert Xu wrote:
> > 
> > > On Sun, Feb 06, 2011 at 08:43:22PM +0100, Jesper Juhl wrote:
> > > > 
> > > > Herbert: If Tadeusz agrees, could you please replace the patch
> > > > you merged 
> > > > with the one above?
> > > 
> > > Please send an incremental patch.
> > > 
> > Sure thing. What would you like it based on exactly?
> 
> The current cryptodev-2.6 tree should do.
> 
Here goes.

Fix up previous patch that attempted to fix a mem leak in 
rfc4106_set_hash_subkey. The previous patch was flawed in that the 'goto 
out' would still leak. Now with sign-offs.

Signed-off-by: Tadeusz Struk <tadeusz.struk@intel.com>
Signed-off-by: Gabriele Paoloni <gabriele.paoloni@intel.com>
---
 arch/x86/crypto/aesni-intel_glue.c |    7 +++----
 1 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c
index e013552..c9f0227 100644
--- a/arch/x86/crypto/aesni-intel_glue.c
+++ b/arch/x86/crypto/aesni-intel_glue.c
@@ -874,11 +874,11 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
 
 	ret = crypto_ablkcipher_setkey(ctr_tfm, key, key_len);
 	if (ret)
-		goto out;
+		goto out_free_ablkcipher;
 
 	req = ablkcipher_request_alloc(ctr_tfm, GFP_KERNEL);
 	if (!req) {
-		ret = -EINVAL;
+		ret = -ENOMEM;
 		goto out_free_ablkcipher;
 	}
 
@@ -911,12 +911,11 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
 		if (!ret)
 			ret = req_data->result.err;
 	}
+	kfree(req_data);
 out_free_request:
 	ablkcipher_request_free(req);
-	kfree(req_data);
 out_free_ablkcipher:
 	crypto_free_ablkcipher(ctr_tfm);
-out:
 	return ret;
 }
 
-- 
1.7.4

--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

[PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().

[Jesper Juhl]

There's a small memory leak in 
arch/x86/crypto/aesni-intel_glue.c::rfc4106_set_hash_subkey(). If the call 
to kmalloc() fails and returns NULL then the memory allocated previously 
by ablkcipher_request_alloc() is not freed when we leave the function.

I could have just added a call to ablkcipher_request_free() before we 
return -ENOMEM, but that started to look too much like the code we 
already had at the end of the function, so I chose instead to rework the 
code a bit so that there are now a few labels at the end that we goto when 
various allocations fail, so we don't have to repeat the same blocks of 
code (this also reduces the object code size slightly).

Please review and consider merging.

Signed-off-by: Jesper Juhl <jj@chaosbits.net>
---
 aesni-intel_glue.c |   17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)

  compile tested only.

diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c
index e1e60c7..e013552 100644
--- a/arch/x86/crypto/aesni-intel_glue.c
+++ b/arch/x86/crypto/aesni-intel_glue.c
@@ -873,21 +873,19 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
 	crypto_ablkcipher_clear_flags(ctr_tfm, ~0);
 
 	ret = crypto_ablkcipher_setkey(ctr_tfm, key, key_len);
-	if (ret) {
-		crypto_free_ablkcipher(ctr_tfm);
-		return ret;
-	}
+	if (ret)
+		goto out;
 
 	req = ablkcipher_request_alloc(ctr_tfm, GFP_KERNEL);
 	if (!req) {
-		crypto_free_ablkcipher(ctr_tfm);
-		return -EINVAL;
+		ret = -EINVAL;
+		goto out_free_ablkcipher;
 	}
 
 	req_data = kmalloc(sizeof(*req_data), GFP_KERNEL);
 	if (!req_data) {
-		crypto_free_ablkcipher(ctr_tfm);
-		return -ENOMEM;
+		ret = -ENOMEM;
+		goto out_free_request;
 	}
 	memset(req_data->iv, 0, sizeof(req_data->iv));
 
@@ -913,9 +911,12 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
 		if (!ret)
 			ret = req_data->result.err;
 	}
+out_free_request:
 	ablkcipher_request_free(req);
 	kfree(req_data);
+out_free_ablkcipher:
 	crypto_free_ablkcipher(ctr_tfm);
+out:
 	return ret;
 }
 


-- 
Jesper Juhl <jj@chaosbits.net>            http://www.chaosbits.net/
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please.

RE: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().

[Struk, Tadeusz]

Hi Jesper,
We have tested your changes and all is fine.
Thank you for identifying this issue.
Regards,
Tadeusz

-----Original Message-----
From: Jesper Juhl [mailto:jj@chaosbits.net] 
Sent: Sunday, January 16, 2011 2:39 PM
To: linux-crypto@vger.kernel.org
Cc: Struk, Tadeusz; linux-kernel@vger.kernel.org; x86@kernel.org; H. Peter Anvin; Ingo Molnar; Thomas Gleixner; David S. Miller; Herbert Xu; Huang, Ying; Hoban, Adrian; Paoloni, Gabriele; O Mahony, Aidan
Subject: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().

There's a small memory leak in 
arch/x86/crypto/aesni-intel_glue.c::rfc4106_set_hash_subkey(). If the call 
to kmalloc() fails and returns NULL then the memory allocated previously 
by ablkcipher_request_alloc() is not freed when we leave the function.

I could have just added a call to ablkcipher_request_free() before we 
return -ENOMEM, but that started to look too much like the code we 
already had at the end of the function, so I chose instead to rework the 
code a bit so that there are now a few labels at the end that we goto when 
various allocations fail, so we don't have to repeat the same blocks of 
code (this also reduces the object code size slightly).

Please review and consider merging.

Signed-off-by: Jesper Juhl <jj@chaosbits.net>
---
 aesni-intel_glue.c |   17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)

  compile tested only.

diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c
index e1e60c7..e013552 100644
--- a/arch/x86/crypto/aesni-intel_glue.c
+++ b/arch/x86/crypto/aesni-intel_glue.c
@@ -873,21 +873,19 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
 	crypto_ablkcipher_clear_flags(ctr_tfm, ~0);
 
 	ret = crypto_ablkcipher_setkey(ctr_tfm, key, key_len);
-	if (ret) {
-		crypto_free_ablkcipher(ctr_tfm);
-		return ret;
-	}
+	if (ret)
+		goto out;
 
 	req = ablkcipher_request_alloc(ctr_tfm, GFP_KERNEL);
 	if (!req) {
-		crypto_free_ablkcipher(ctr_tfm);
-		return -EINVAL;
+		ret = -EINVAL;
+		goto out_free_ablkcipher;
 	}
 
 	req_data = kmalloc(sizeof(*req_data), GFP_KERNEL);
 	if (!req_data) {
-		crypto_free_ablkcipher(ctr_tfm);
-		return -ENOMEM;
+		ret = -ENOMEM;
+		goto out_free_request;
 	}
 	memset(req_data->iv, 0, sizeof(req_data->iv));
 
@@ -913,9 +911,12 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
 		if (!ret)
 			ret = req_data->result.err;
 	}
+out_free_request:
 	ablkcipher_request_free(req);
 	kfree(req_data);
+out_free_ablkcipher:
 	crypto_free_ablkcipher(ctr_tfm);
+out:
 	return ret;
 }
 


-- 
Jesper Juhl <jj@chaosbits.net>            http://www.chaosbits.net/
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html
Plain text mails only, please.

--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.

RE: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().

[Jesper Juhl]

On Wed, 19 Jan 2011, Struk, Tadeusz wrote:

> Hi Jesper,

Hi Tadeusz,

> We have tested your changes and all is fine.
> Thank you for identifying this issue.

You are welcome. But I believe I made a mistake in that patch. I looked at 
it again and I'm convinced that the 'out' label should not exist. The 
jump to it will still leak 'ctr_tfm' - don't you agree?

I believe the patch should look like this instead:


Signed-off-by: Jesper Juhl <jj@chaosbits.net>
---
 aesni-intel_glue.c |   16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c
index e1e60c7..4a8c015 100644
--- a/arch/x86/crypto/aesni-intel_glue.c
+++ b/arch/x86/crypto/aesni-intel_glue.c
@@ -873,21 +873,19 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
 	crypto_ablkcipher_clear_flags(ctr_tfm, ~0);
 
 	ret = crypto_ablkcipher_setkey(ctr_tfm, key, key_len);
-	if (ret) {
-		crypto_free_ablkcipher(ctr_tfm);
-		return ret;
-	}
+	if (ret)
+		goto out_free_ablkcipher;
 
 	req = ablkcipher_request_alloc(ctr_tfm, GFP_KERNEL);
 	if (!req) {
-		crypto_free_ablkcipher(ctr_tfm);
-		return -EINVAL;
+		ret = -EINVAL;
+		goto out_free_ablkcipher;
 	}
 
 	req_data = kmalloc(sizeof(*req_data), GFP_KERNEL);
 	if (!req_data) {
-		crypto_free_ablkcipher(ctr_tfm);
-		return -ENOMEM;
+		ret = -ENOMEM;
+		goto out_free_request;
 	}
 	memset(req_data->iv, 0, sizeof(req_data->iv));
 
@@ -913,8 +911,10 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
 		if (!ret)
 			ret = req_data->result.err;
 	}
+out_free_request:
 	ablkcipher_request_free(req);
 	kfree(req_data);
+out_free_ablkcipher:
 	crypto_free_ablkcipher(ctr_tfm);
 	return ret;
 }


Herbert: If Tadeusz agrees, could you please replace the patch you merged 
with the one above?

-- 
Jesper Juhl <jj@chaosbits.net>            http://www.chaosbits.net/
Plain text mails only, please.
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html

Re: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().

[Herbert Xu]

On Sun, Feb 06, 2011 at 08:43:22PM +0100, Jesper Juhl wrote:
> 
> Herbert: If Tadeusz agrees, could you please replace the patch you merged 
> with the one above?

Please send an incremental patch.

Thanks,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().

[Jesper Juhl]

On Mon, 7 Feb 2011, Herbert Xu wrote:

> On Sun, Feb 06, 2011 at 08:43:22PM +0100, Jesper Juhl wrote:
> > 
> > Herbert: If Tadeusz agrees, could you please replace the patch you merged 
> > with the one above?
> 
> Please send an incremental patch.
> 
Sure thing. What would you like it based on exactly?

-- 
Jesper Juhl <jj@chaosbits.net>            http://www.chaosbits.net/
Plain text mails only, please.
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html

Re: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().

[Herbert Xu]

On Sun, Feb 06, 2011 at 09:34:33PM +0100, Jesper Juhl wrote:
> On Mon, 7 Feb 2011, Herbert Xu wrote:
> 
> > On Sun, Feb 06, 2011 at 08:43:22PM +0100, Jesper Juhl wrote:
> > > 
> > > Herbert: If Tadeusz agrees, could you please replace the patch you merged 
> > > with the one above?
> > 
> > Please send an incremental patch.
> > 
> Sure thing. What would you like it based on exactly?

The current cryptodev-2.6 tree should do.

Cheers,
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

Re: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().

[Jesper Juhl]

On Mon, 7 Feb 2011, Herbert Xu wrote:

> On Sun, Feb 06, 2011 at 09:34:33PM +0100, Jesper Juhl wrote:
> > On Mon, 7 Feb 2011, Herbert Xu wrote:
> > 
> > > On Sun, Feb 06, 2011 at 08:43:22PM +0100, Jesper Juhl wrote:
> > > > 
> > > > Herbert: If Tadeusz agrees, could you please replace the patch you merged 
> > > > with the one above?
> > > 
> > > Please send an incremental patch.
> > > 
> > Sure thing. What would you like it based on exactly?
> 
> The current cryptodev-2.6 tree should do.
> 
Here goes.

Fix up previous patch that attempted to fix a mem leak in 
rfc4106_set_hash_subkey. The previous patch was flawed in that the 'goto 
out' would still leak.

Signed-off-by: Jesper Juhl <jj@chaosbits.net>
---
 aesni-intel_glue.c |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git 
a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c
index e013552..4a8c015 100644
--- a/arch/x86/crypto/aesni-intel_glue.c
+++ b/arch/x86/crypto/aesni-intel_glue.c
@@ -874,7 +874,7 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
 
 	ret = crypto_ablkcipher_setkey(ctr_tfm, key, key_len);
 	if (ret)
-		goto out;
+		goto out_free_ablkcipher;
 
 	req = ablkcipher_request_alloc(ctr_tfm, GFP_KERNEL);
 	if (!req) {
@@ -916,7 +916,6 @@ out_free_request:
 	kfree(req_data);
 out_free_ablkcipher:
 	crypto_free_ablkcipher(ctr_tfm);
-out:
 	return ret;
 }
 
 

-- 
Jesper Juhl <jj@chaosbits.net>            http://www.chaosbits.net/
Plain text mails only, please.
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html

Re: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().

[Herbert Xu]

On Sun, Jan 16, 2011 at 03:38:47PM +0100, Jesper Juhl wrote:
> There's a small memory leak in 
> arch/x86/crypto/aesni-intel_glue.c::rfc4106_set_hash_subkey(). If the call 
> to kmalloc() fails and returns NULL then the memory allocated previously 
> by ablkcipher_request_alloc() is not freed when we leave the function.
> 
> I could have just added a call to ablkcipher_request_free() before we 
> return -ENOMEM, but that started to look too much like the code we 
> already had at the end of the function, so I chose instead to rework the 
> code a bit so that there are now a few labels at the end that we goto when 
> various allocations fail, so we don't have to repeat the same blocks of 
> code (this also reduces the object code size slightly).
> 
> Please review and consider merging.
> 
> Signed-off-by: Jesper Juhl <jj@chaosbits.net>

Patch applied.  Thanks a lot!
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt