LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* Re: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().
@ 2011-02-07 17:32 tadeusz.struk
  2011-02-07 18:24 ` Jesper Juhl
  2011-02-07 21:09 ` [PATCH] " Herbert Xu
  0 siblings, 2 replies; 10+ messages in thread
From: tadeusz.struk @ 2011-02-07 17:32 UTC (permalink / raw)
  To: herbert
  Cc: linux-kernel, linux-crypto, aidan.o.mahony, gabriele.paoloni,
	adrian.hoban, tadeusz.struk, jj

From: 
Date: Sun, 16 Jan 2011 16:41:11 +0000
Subject: RE: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().

Hi Jesper,
Thanks, but I think there is still a problem here. You don't want to kfree req_data
when the kmalloc failed. I think it should look as follows.  
Are you ok with this?

On Mon, 7 Feb 2011, Herbert Xu wrote:

> On Sun, Feb 06, 2011 at 09:34:33PM +0100, Jesper Juhl wrote:
> > On Mon, 7 Feb 2011, Herbert Xu wrote:
> > 
> > > On Sun, Feb 06, 2011 at 08:43:22PM +0100, Jesper Juhl wrote:
> > > > 
> > > > Herbert: If Tadeusz agrees, could you please replace the patch
> > > > you merged 
> > > > with the one above?
> > > 
> > > Please send an incremental patch.
> > > 
> > Sure thing. What would you like it based on exactly?
> 
> The current cryptodev-2.6 tree should do.
> 
Here goes.

Fix up previous patch that attempted to fix a mem leak in 
rfc4106_set_hash_subkey. The previous patch was flawed in that the 'goto 
out' would still leak.

---
 arch/x86/crypto/aesni-intel_glue.c |    7 +++----
 1 files changed, 3 insertions(+), 4 deletions(-)

diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c
index e013552..c9f0227 100644
--- a/arch/x86/crypto/aesni-intel_glue.c
+++ b/arch/x86/crypto/aesni-intel_glue.c
@@ -874,11 +874,11 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
 
 	ret = crypto_ablkcipher_setkey(ctr_tfm, key, key_len);
 	if (ret)
-		goto out;
+		goto out_free_ablkcipher;
 
 	req = ablkcipher_request_alloc(ctr_tfm, GFP_KERNEL);
 	if (!req) {
-		ret = -EINVAL;
+		ret = -ENOMEM;
 		goto out_free_ablkcipher;
 	}
 
@@ -911,12 +911,11 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
 		if (!ret)
 			ret = req_data->result.err;
 	}
+	kfree(req_data);
 out_free_request:
 	ablkcipher_request_free(req);
-	kfree(req_data);
 out_free_ablkcipher:
 	crypto_free_ablkcipher(ctr_tfm);
-out:
 	return ret;
 }
 
-- 
1.7.4

--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.



^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().
  2011-02-07 17:32 [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey() tadeusz.struk
@ 2011-02-07 18:24 ` Jesper Juhl
  2011-02-11 14:26   ` Pavel Machek
  2011-02-07 21:09 ` [PATCH] " Herbert Xu
  1 sibling, 1 reply; 10+ messages in thread
From: Jesper Juhl @ 2011-02-07 18:24 UTC (permalink / raw)
  To: tadeusz.struk
  Cc: herbert, linux-kernel, linux-crypto, aidan.o.mahony,
	gabriele.paoloni, adrian.hoban

On Mon, 7 Feb 2011, tadeusz.struk@intel.com wrote:

> From: 
> Date: Sun, 16 Jan 2011 16:41:11 +0000
> Subject: RE: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().
> 
> Hi Jesper,
> Thanks, but I think there is still a problem here. You don't want to kfree req_data
> when the kmalloc failed. I think it should look as follows.  
> Are you ok with this?
> 
Fine by me.

I was aware of the kfree(NULL) thing, but desided to leave it as is for 
two reasons - 1) kfree(NULL) is harmless and this is an error path, so the 
extra function call doesn't matter much. 2) I wanted to preserve 
deallocations in the reverse order of the allocations. But sure, moving 
that kfree is a tiny optimization of the error path, so I'm fine with it.


-- 
Jesper Juhl <jj@chaosbits.net>            http://www.chaosbits.net/
Plain text mails only, please.
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().
  2011-02-07 17:32 [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey() tadeusz.struk
  2011-02-07 18:24 ` Jesper Juhl
@ 2011-02-07 21:09 ` Herbert Xu
  2011-02-10 18:47   ` Jesper Juhl
  1 sibling, 1 reply; 10+ messages in thread
From: Herbert Xu @ 2011-02-07 21:09 UTC (permalink / raw)
  To: tadeusz.struk
  Cc: linux-kernel, linux-crypto, aidan.o.mahony, gabriele.paoloni,
	adrian.hoban, jj

On Mon, Feb 07, 2011 at 05:32:59PM +0000, tadeusz.struk@intel.com wrote:
>
> Here goes.
> 
> Fix up previous patch that attempted to fix a mem leak in 
> rfc4106_set_hash_subkey. The previous patch was flawed in that the 'goto 
> out' would still leak.

Sign-off?
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().
  2011-02-07 21:09 ` [PATCH] " Herbert Xu
@ 2011-02-10 18:47   ` Jesper Juhl
  0 siblings, 0 replies; 10+ messages in thread
From: Jesper Juhl @ 2011-02-10 18:47 UTC (permalink / raw)
  To: Herbert Xu
  Cc: tadeusz.struk, linux-kernel, linux-crypto, aidan.o.mahony,
	gabriele.paoloni, adrian.hoban

On Tue, 8 Feb 2011, Herbert Xu wrote:

> On Mon, Feb 07, 2011 at 05:32:59PM +0000, tadeusz.struk@intel.com wrote:
> >
> > Here goes.
> > 
> > Fix up previous patch that attempted to fix a mem leak in 
> > rfc4106_set_hash_subkey. The previous patch was flawed in that the 'goto 
> > out' would still leak.
> 
> Sign-off?
> 

Since it's identical to the patch I posted except for the moving of the 
kfree() which (IMHO) is a minor detail, I hope you can use my sign-off so 
we can move along and get the patch merged..

Signed-off-by: Jesper Juhl <jj@chaosbits.net>


-- 
Jesper Juhl <jj@chaosbits.net>            http://www.chaosbits.net/
Plain text mails only, please.
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().
  2011-02-07 18:24 ` Jesper Juhl
@ 2011-02-11 14:26   ` Pavel Machek
  2011-02-11 14:38     ` Paoloni, Gabriele
  0 siblings, 1 reply; 10+ messages in thread
From: Pavel Machek @ 2011-02-11 14:26 UTC (permalink / raw)
  To: Jesper Juhl
  Cc: tadeusz.struk, herbert, linux-kernel, linux-crypto,
	aidan.o.mahony, gabriele.paoloni, adrian.hoban

On Mon 2011-02-07 19:24:43, Jesper Juhl wrote:
> On Mon, 7 Feb 2011, tadeusz.struk@intel.com wrote:
> 
> > From: 
> > Date: Sun, 16 Jan 2011 16:41:11 +0000
> > Subject: RE: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().
> > 
> > Hi Jesper,
> > Thanks, but I think there is still a problem here. You don't want to kfree req_data
> > when the kmalloc failed. I think it should look as follows.  
> > Are you ok with this?
> > 
> Fine by me.
> 
> I was aware of the kfree(NULL) thing, but desided to leave it as is for 
> two reasons - 1) kfree(NULL) is harmless and this is an error path, so the 
> extra function call doesn't matter much. 2) I wanted to preserve 
> deallocations in the reverse order of the allocations. But sure, moving 
> that kfree is a tiny optimization of the error path, so I'm fine with it.

I don't think such optimalization is worth doing... original code is
as good but shorter and less complex...

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

^ permalink raw reply	[flat|nested] 10+ messages in thread

* RE: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().
  2011-02-11 14:26   ` Pavel Machek
@ 2011-02-11 14:38     ` Paoloni, Gabriele
  2011-02-11 14:47       ` Jesper Juhl
  0 siblings, 1 reply; 10+ messages in thread
From: Paoloni, Gabriele @ 2011-02-11 14:38 UTC (permalink / raw)
  To: Pavel Machek, Jesper Juhl
  Cc: Struk, Tadeusz, herbert, linux-kernel, linux-crypto, O Mahony,
	Aidan, Hoban, Adrian

Well anyway I think that the return value of "ablkcipher_request_alloc(ctr_tfm, GFP_KERNEL)" has to be changed from -EINVAL to -ENOMEM in case of failure. That is why would stay on the patch that Tadeusz proposed. Otherwise Juhl should send another one....

> -----Original Message-----
> From: Pavel Machek [mailto:pavel@ucw.cz]
> Sent: Friday, February 11, 2011 2:27 PM
> To: Jesper Juhl
> Cc: Struk, Tadeusz; herbert@gondor.hengli.com.au; linux-
> kernel@vger.kernel.org; linux-crypto@vger.kernel.org; O Mahony, Aidan;
> Paoloni, Gabriele; Hoban, Adrian
> Subject: Re: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in
> rfc4106_set_hash_subkey().
> 
> On Mon 2011-02-07 19:24:43, Jesper Juhl wrote:
> > On Mon, 7 Feb 2011, tadeusz.struk@intel.com wrote:
> >
> > > From:
> > > Date: Sun, 16 Jan 2011 16:41:11 +0000
> > > Subject: RE: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in
> rfc4106_set_hash_subkey().
> > >
> > > Hi Jesper,
> > > Thanks, but I think there is still a problem here. You don't want
> to kfree req_data
> > > when the kmalloc failed. I think it should look as follows.
> > > Are you ok with this?
> > >
> > Fine by me.
> >
> > I was aware of the kfree(NULL) thing, but desided to leave it as is
> for
> > two reasons - 1) kfree(NULL) is harmless and this is an error path,
> so the
> > extra function call doesn't matter much. 2) I wanted to preserve
> > deallocations in the reverse order of the allocations. But sure,
> moving
> > that kfree is a tiny optimization of the error path, so I'm fine with
> it.
> 
> I don't think such optimalization is worth doing... original code is
> as good but shorter and less complex...
> 
> --
> (english) http://www.livejournal.com/~pavelmachek
> (cesky, pictures)
> http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html
--------------------------------------------------------------
Intel Shannon Limited
Registered in Ireland
Registered Office: Collinstown Industrial Park, Leixlip, County Kildare
Registered Number: 308263
Business address: Dromore House, East Park, Shannon, Co. Clare

This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.



^ permalink raw reply	[flat|nested] 10+ messages in thread

* RE: [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey().
  2011-02-11 14:38     ` Paoloni, Gabriele
@ 2011-02-11 14:47       ` Jesper Juhl
  2011-02-11 21:09         ` [PATCH](updated) " Jesper Juhl
  0 siblings, 1 reply; 10+ messages in thread
From: Jesper Juhl @ 2011-02-11 14:47 UTC (permalink / raw)
  To: Paoloni, Gabriele
  Cc: Pavel Machek, Struk, Tadeusz, herbert, linux-kernel,
	linux-crypto, O Mahony, Aidan, Hoban, Adrian

On Fri, 11 Feb 2011, Paoloni, Gabriele wrote:

> Well anyway I think that the return value of "ablkcipher_request_alloc(ctr_tfm, GFP_KERNEL)" has to be changed from -EINVAL to -ENOMEM in case of failure. That is why would stay on the patch that Tadeusz proposed. Otherwise Juhl should send another one....
> 
I'll take a look again later this evening when I get home from work.


-- 
Jesper Juhl <jj@chaosbits.net>            http://www.chaosbits.net/
Plain text mails only, please.
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html


^ permalink raw reply	[flat|nested] 10+ messages in thread

* RE: [PATCH](updated) rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey()..
  2011-02-11 14:47       ` Jesper Juhl
@ 2011-02-11 21:09         ` Jesper Juhl
  2011-02-11 21:14           ` Jesper Juhl
  0 siblings, 1 reply; 10+ messages in thread
From: Jesper Juhl @ 2011-02-11 21:09 UTC (permalink / raw)
  To: Paoloni, Gabriele, Herbert Xu, tadeusz.struk
  Cc: Pavel Machek, linux-kernel, linux-crypto, O Mahony, Aidan, Hoban, Adrian

On Fri, 11 Feb 2011, Jesper Juhl wrote:

> On Fri, 11 Feb 2011, Paoloni, Gabriele wrote:
> 
> > Well anyway I think that the return value of "ablkcipher_request_alloc(ctr_tfm, GFP_KERNEL)" has to be changed from -EINVAL to -ENOMEM in case of failure. That is why would stay on the patch that Tadeusz proposed. Otherwise Juhl should send another one....
> > 
> I'll take a look again later this evening when I get home from work.
> 
Hopefully this takes care of all complaints and can actually get merged so 
we can get the leak fixed (patch is against current cryptodev-2.6 tree).


Fix up previous patch that failed to properly fix mem leak in 
rfc4106_set_hash_subkey(). This add-on patch; fixes the leak. moves 
kfree() out of the error path, returns -ENOMEM rather than -EINVAL when 
ablkcipher_request_alloc() fails.

Signed-off-by: Jesper Juhl <jj@chaosbits.net>
---
 aesni-intel_glue.c |   13 +++++--------
 1 file changed, 5 insertions(+), 8 deletions(-)

diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c
index e013552..502b76d 100644
--- a/arch/x86/crypto/aesni-intel_glue.c
+++ b/arch/x86/crypto/aesni-intel_glue.c
@@ -876,17 +876,15 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
 	if (ret)
 		goto out;
 
+	ret = -ENOMEM;
 	req = ablkcipher_request_alloc(ctr_tfm, GFP_KERNEL);
-	if (!req) {
-		ret = -EINVAL;
+	if (!req)
 		goto out_free_ablkcipher;
-	}
 
 	req_data = kmalloc(sizeof(*req_data), GFP_KERNEL);
-	if (!req_data) {
-		ret = -ENOMEM;
+	if (!req_data)
 		goto out_free_request;
-	}
+
 	memset(req_data->iv, 0, sizeof(req_data->iv));
 
 	/* Clear the data in the hash sub key container to zero.*/
@@ -911,12 +909,11 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
 		if (!ret)
 			ret = req_data->result.err;
 	}
+	kfree(req_data);
 out_free_request:
 	ablkcipher_request_free(req);
-	kfree(req_data);
 out_free_ablkcipher:
 	crypto_free_ablkcipher(ctr_tfm);
-out:
 	return ret;
 }
 


-- 
Jesper Juhl <jj@chaosbits.net>            http://www.chaosbits.net/
Plain text mails only, please.
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html


^ permalink raw reply	[flat|nested] 10+ messages in thread

* RE: [PATCH](updated) rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey()..
  2011-02-11 21:09         ` [PATCH](updated) " Jesper Juhl
@ 2011-02-11 21:14           ` Jesper Juhl
  2011-02-16  2:04             ` Herbert Xu
  0 siblings, 1 reply; 10+ messages in thread
From: Jesper Juhl @ 2011-02-11 21:14 UTC (permalink / raw)
  To: Paoloni, Gabriele, Herbert Xu, tadeusz.struk
  Cc: Pavel Machek, linux-kernel, linux-crypto, O Mahony, Aidan, Hoban, Adrian

On Fri, 11 Feb 2011, Jesper Juhl wrote:

> On Fri, 11 Feb 2011, Jesper Juhl wrote:
> 
> > On Fri, 11 Feb 2011, Paoloni, Gabriele wrote:
> > 
> > > Well anyway I think that the return value of "ablkcipher_request_alloc(ctr_tfm, GFP_KERNEL)" has to be changed from -EINVAL to -ENOMEM in case of failure. That is why would stay on the patch that Tadeusz proposed. Otherwise Juhl should send another one....
> > > 
> > I'll take a look again later this evening when I get home from work.
> > 
> Hopefully this takes care of all complaints and can actually get merged so 
> we can get the leak fixed (patch is against current cryptodev-2.6 tree).
> 
> 
> Fix up previous patch that failed to properly fix mem leak in 
> rfc4106_set_hash_subkey(). This add-on patch; fixes the leak. moves 
> kfree() out of the error path, returns -ENOMEM rather than -EINVAL when 
> ablkcipher_request_alloc() fails.
> 

And of course I just had to screw up and send the wrong diff. The one I 
sent does not compile and was a temporary file I imported into the mail by 
accident :-(
Below is the real patch - changelog above still applies - sorry about 
that.

Signed-off-by: Jesper Juhl <jj@chaosbits.net>
---
 aesni-intel_glue.c |   15 ++++++---------
 1 file changed, 6 insertions(+), 9 deletions(-)

diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c
index e013552..e0e6340 100644
--- a/arch/x86/crypto/aesni-intel_glue.c
+++ b/arch/x86/crypto/aesni-intel_glue.c
@@ -874,19 +874,17 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
 
 	ret = crypto_ablkcipher_setkey(ctr_tfm, key, key_len);
 	if (ret)
-		goto out;
+		goto out_free_ablkcipher;
 
+	ret = -ENOMEM;
 	req = ablkcipher_request_alloc(ctr_tfm, GFP_KERNEL);
-	if (!req) {
-		ret = -EINVAL;
+	if (!req)
 		goto out_free_ablkcipher;
-	}
 
 	req_data = kmalloc(sizeof(*req_data), GFP_KERNEL);
-	if (!req_data) {
-		ret = -ENOMEM;
+	if (!req_data)
 		goto out_free_request;
-	}
+
 	memset(req_data->iv, 0, sizeof(req_data->iv));
 
 	/* Clear the data in the hash sub key container to zero.*/
@@ -911,12 +909,11 @@ rfc4106_set_hash_subkey(u8 *hash_subkey, const u8 *key, unsigned int key_len)
 		if (!ret)
 			ret = req_data->result.err;
 	}
+	kfree(req_data);
 out_free_request:
 	ablkcipher_request_free(req);
-	kfree(req_data);
 out_free_ablkcipher:
 	crypto_free_ablkcipher(ctr_tfm);
-out:
 	return ret;
 }
 
 
-- 
Jesper Juhl <jj@chaosbits.net>            http://www.chaosbits.net/
Plain text mails only, please.
Don't top-post http://www.catb.org/~esr/jargon/html/T/top-post.html


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: [PATCH](updated) rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey()..
  2011-02-11 21:14           ` Jesper Juhl
@ 2011-02-16  2:04             ` Herbert Xu
  0 siblings, 0 replies; 10+ messages in thread
From: Herbert Xu @ 2011-02-16  2:04 UTC (permalink / raw)
  To: Jesper Juhl
  Cc: Paoloni, Gabriele, tadeusz.struk, Pavel Machek, linux-kernel,
	linux-crypto, O Mahony, Aidan, Hoban, Adrian

On Fri, Feb 11, 2011 at 10:14:44PM +0100, Jesper Juhl wrote:
> 
> And of course I just had to screw up and send the wrong diff. The one I 
> sent does not compile and was a temporary file I imported into the mail by 
> accident :-(
> Below is the real patch - changelog above still applies - sorry about 
> that.
> 
> Signed-off-by: Jesper Juhl <jj@chaosbits.net>

Patch applied.  Thanks!
-- 
Email: Herbert Xu <herbert@gondor.apana.org.au>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2011-02-16  2:04 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2011-02-07 17:32 [PATCH] rfc4106, Intel, AES-NI: Don't leak memory in rfc4106_set_hash_subkey() tadeusz.struk
2011-02-07 18:24 ` Jesper Juhl
2011-02-11 14:26   ` Pavel Machek
2011-02-11 14:38     ` Paoloni, Gabriele
2011-02-11 14:47       ` Jesper Juhl
2011-02-11 21:09         ` [PATCH](updated) " Jesper Juhl
2011-02-11 21:14           ` Jesper Juhl
2011-02-16  2:04             ` Herbert Xu
2011-02-07 21:09 ` [PATCH] " Herbert Xu
2011-02-10 18:47   ` Jesper Juhl

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).