[Classpathx-javamail] Parsing a MimeMultipart

[remi__]

Hi,

I am using the classpathx-mail implentation of Javamail through JOnAS 4.8.4.
It is the version 1.1.1
I actually have a problem parsing a mail with a multipart content.

Here is the multipart section of the original message (mail retrieved using
the IMAP protocol). I dropped the routing information and just display the
mulipart section:

Content-Type: multipart/mixed;
boundary="----=_Part_0_12966337.1179326080592"
------=_Part_0_12966337.1179326080592
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit

------=_Part_0_12966337.1179326080592
Content-Type: text/plain;
name=testListeMultiPJ_16052007_163439_0157_Z4W897.xml
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=testListeMultiPJ_16052007_163439_0157_Z4W897.xml

<?xml version="1.0" encoding="UTF-8"?>
<test> content ...... </test>
------=_Part_0_12966337.1179326080592

I cast the message content to a Multipart :
Multipart piecesMessage = (Multipart) message.getContent();

Here is what I get if I display the bodypart related to the xml file:

Content-Type: text/plain;
name=testListeMultiPJ_16052007_164150_0159_LG7FFCQU7AUKN3XFW
Content-Transfer-Encoding: 2
Content-Id: 7bit
Content-Description: 558

It seems that the parser does not give the right values to the respective
content-headers.

Is there any bug in the implementation or am I using the library in a wrong
way?

NB : the code works with the SUN implementation of Javamail...

Thansk for your support

[Chris Burdess]

Post by remi__
I am using the classpathx-mail implentation of Javamail through JOnAS 4.8.4.
It is the version 1.1.1
I actually have a problem parsing a mail with a multipart content.
Here is the multipart section of the original message (mail
retrieved using
the IMAP protocol). I dropped the routing information and just
display the
Content-Type: multipart/mixed;
boundary="----=_Part_0_12966337.1179326080592"
------=_Part_0_12966337.1179326080592
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
------=_Part_0_12966337.1179326080592
Content-Type: text/plain;
name=testListeMultiPJ_16052007_163439_0157_Z4W897.xml
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=testListeMultiPJ_16052007_163439_0157_Z4W897.xml
<?xml version="1.0" encoding="UTF-8"?>
<test> content ...... </test>
------=_Part_0_12966337.1179326080592
Multipart piecesMessage = (Multipart) message.getContent();
Content-Type: text/plain;
name=testListeMultiPJ_16052007_164150_0159_LG7FFCQU7AUKN3XFW
Content-Transfer-Encoding: 2
Content-Id: 7bit
Content-Description: 558
It seems that the parser does not give the right values to the
respective
content-headers.
Is there any bug in the implementation or am I using the library in a wrong
way?

There may be a bug, it's pretty difficult for me to tell or not from
the information you're giving me.

If you can provide a test case, it would help to determine what the
issue is.

[remi__]

Hej,

Here is a test case.
I added the client.jar file from JOnAS 4.8.4 lib to the classpath to compile
and run. This jar contains a javamail implementation based on GNU-javamail
1.1.1

The test case sends a mail to an IMAP server with a few xml enclosed files
(you have to specify the file in the code). It retrieves the mail and
displays one of the body parts.

The output is done on System.out and the result displays the "bug" ( if it
is a bug ...)

Thanks

Rémi

http://www.nabble.com/file/8463/TestGNUMail.java TestGNUMail.java

Post by Chris Burdess

Post by remi__
I am using the classpathx-mail implentation of Javamail through JOnAS 4.8.4.
It is the version 1.1.1
I actually have a problem parsing a mail with a multipart content.
It seems that the parser does not give the right values to the respective
content-headers.
Is there any bug in the implementation or am I using the library in a wrong
way?

There may be a bug, it's pretty difficult for me to tell or not from
the information you're giving me.
If you can provide a test case, it would help to determine what the
issue is.
_______________________________________________
Classpathx-javamail mailing list
http://lists.gnu.org/mailman/listinfo/classpathx-javamail

[Chris Burdess]

Post by remi__
Here is a test case.
I added the client.jar file from JOnAS 4.8.4 lib to the classpath to compile
and run. This jar contains a javamail implementation based on GNU-
javamail
1.1.1
The test case sends a mail to an IMAP server with a few xml
enclosed files
(you have to specify the file in the code). It retrieves the mail and
displays one of the body parts.
The output is done on System.out and the result displays the
"bug" ( if it
is a bug ...)
Thanks
Rémi
http://www.nabble.com/file/8463/TestGNUMail.java TestGNUMail.java

Thanks for this.

I just tried out your test here, this is the result (tested against
an Exchange server):

Content-Type: application/octet-stream; name=file.xml
Content-Transfer-Encoding: 7BIT
Content-Disposition: attachment; filename=file.xml

<test/>

So this leads me to suspect something weird in the specific server
you're using. Perhaps you could capture the TCP session using a tool
like tcpflow for the duration of your program?

[remi__]

Thanks very much for having run the test.

I ran the test using a hmailserver. That might be the reason of my problem.
However, it is pretty weird that, when I call message.writeTo(System.out) on
the Message (or MimeMessage...) instance that I receive, it prints a message
with the right headers. The headers become strange when I call the
getContent() and getBodyPart(..) methods.
I am going to run some more tests.

By the way, did you use the client.jar lib from JOnAS or did you directly
use the GNU-mail lib ?

Have a nice day,

Rémi

Post by Chris Burdess

Post by remi__
Here is a test case.
I added the client.jar file from JOnAS 4.8.4 lib to the classpath to compile
and run. This jar contains a javamail implementation based on GNU-
javamail
1.1.1
The test case sends a mail to an IMAP server with a few xml
enclosed files
(you have to specify the file in the code). It retrieves the mail and
displays one of the body parts.
The output is done on System.out and the result displays the
"bug" ( if it
is a bug ...)
Thanks
Rémi
http://www.nabble.com/file/8463/TestGNUMail.java TestGNUMail.java

Thanks for this.
I just tried out your test here, this is the result (tested against
Content-Type: application/octet-stream; name=file.xml
Content-Transfer-Encoding: 7BIT
Content-Disposition: attachment; filename=file.xml
<test/>
So this leads me to suspect something weird in the specific server
you're using. Perhaps you could capture the TCP session using a tool
like tcpflow for the duration of your program?
_______________________________________________
Classpathx-javamail mailing list
http://lists.gnu.org/mailman/listinfo/classpathx-javamail

[Chris Burdess]

Post by remi__
By the way, did you use the client.jar lib from JOnAS or did you directly
use the GNU-mail lib ?

I used the gnumail jars directly.

[remi__]

As you suggested, here is the answer of themail server I use.

* 23 FETCH (BODYSTRUCTURE (("TEXT" "PLAIN" ("CHARSET" "windows-1252") NIL
NIL "7bit" 9 1)("TEXT" "PLAIN" ("NAME" "file") NIL NIL "base64" 2516 33 NIL
("attachment" ("FILENAME" "file")))("TEXT" "PLAIN" ("NAME" "file") NIL NIL
"base64" 2516 33 NIL ("attachment" ("FILENAME" "file")))("TEXT" "PLAIN"
("NAME" "file") NIL NIL "base64" 2516 33 NIL ("attachment" ("FILENAME"
"file")))("TEXT" "PLAIN" ("NAME" "file") NIL NIL "base64" 2516 33 NIL
("attachment" ("FILENAME" "file")))("TEXT" "PLAIN" ("NAME" "file") NIL NIL
"base64" 2516 33 NIL ("attachment" ("FILENAME" "file"))) "MIXED" ("BOUNDARY"
"----=_Part_0_31379709.1179480493454") NIL NIL))

And here is the output of my test case:

Content-Type: text/plain; name=file
Content-Transfer-Encoding: 33
javax.mail.MessagingException: Unknown encoding: 33
at javax.mail.internet.MimeUtility.encode(MimeUtility.java:240)
at javax.mail.internet.MimeBodyPart.writeTo(MimeBodyPart.java:804)
at TestGNUMail.displayMail(TestGNUMail.java:61)
at TestGNUMail.test(TestGNUMail.java:45)
at TestGNUMail.main(TestGNUMail.java:33)
Content-Id: base64
Content-Description: 2516

It seems that the "2516 33" pattern is recurrent in the answer. Is that
supposed to be so? I have no knowledge in the IMAP protocol.

I am going to insvestigate in the direction to the server.

Rémi

Post by Chris Burdess

Post by remi__
Here is a test case.
I added the client.jar file from JOnAS 4.8.4 lib to the classpath to compile
and run. This jar contains a javamail implementation based on GNU-
javamail
1.1.1
The test case sends a mail to an IMAP server with a few xml
enclosed files
(you have to specify the file in the code). It retrieves the mail and
displays one of the body parts.
The output is done on System.out and the result displays the
"bug" ( if it
is a bug ...)
Thanks
Rémi
http://www.nabble.com/file/8463/TestGNUMail.java TestGNUMail.java

Thanks for this.
I just tried out your test here, this is the result (tested against
Content-Type: application/octet-stream; name=file.xml
Content-Transfer-Encoding: 7BIT
Content-Disposition: attachment; filename=file.xml
<test/>
So this leads me to suspect something weird in the specific server
you're using. Perhaps you could capture the TCP session using a tool
like tcpflow for the duration of your program?
_______________________________________________
Classpathx-javamail mailing list
http://lists.gnu.org/mailman/listinfo/classpathx-javamail

[Chris Burdess]

Post by remi__
As you suggested, here is the answer of themail server I use.
* 23 FETCH (BODYSTRUCTURE (("TEXT" "PLAIN" ("CHARSET"
"windows-1252") NIL
NIL "7bit" 9 1)("TEXT" "PLAIN" ("NAME" "file") NIL NIL "base64" 2516 33 NIL
("attachment" ("FILENAME" "file")))("TEXT" "PLAIN" ("NAME" "file") NIL NIL
"base64" 2516 33 NIL ("attachment" ("FILENAME" "file")))("TEXT" "PLAIN"
("NAME" "file") NIL NIL "base64" 2516 33 NIL ("attachment" ("FILENAME"
"file")))("TEXT" "PLAIN" ("NAME" "file") NIL NIL "base64" 2516 33 NIL
("attachment" ("FILENAME" "file")))("TEXT" "PLAIN" ("NAME" "file") NIL NIL
"base64" 2516 33 NIL ("attachment" ("FILENAME" "file")))
"MIXED" ("BOUNDARY"
"----=_Part_0_31379709.1179480493454") NIL NIL))
Content-Type: text/plain; name=file
Content-Transfer-Encoding: 33
javax.mail.MessagingException: Unknown encoding: 33
at javax.mail.internet.MimeUtility.encode(MimeUtility.java:240)
at javax.mail.internet.MimeBodyPart.writeTo(MimeBodyPart.java:804)
at TestGNUMail.displayMail(TestGNUMail.java:61)
at TestGNUMail.test(TestGNUMail.java:45)
at TestGNUMail.main(TestGNUMail.java:33)
Content-Id: base64
Content-Description: 2516
It seems that the "2516 33" pattern is recurrent in the answer. Is that
supposed to be so? I have no knowledge in the IMAP protocol.

The response looks mostly normal. It appears to have a content-
disposition in the extension where a content-language is expected,
but that certainly shouldn't break anything and would not cause the
content-id, content-description, and content-transfer-encoding to
have the wrong values.

Maybe the parsing of the IMAP structure is broken at a lower level.
If you have access to a debugging environment, could you post the
value of "list" in the IMAPMessage.parseMultipart method during
execution of your program?

[remi__]

This is the value of "list" just after the call to parseMultipart(List list,
Part parent, InternetHeaders parentHeaders,
String baseSection)

[
[TEXT, PLAIN, [CHARSET, windows-1252], NIL, NIL, 7bit, 9, 1],
[TEXT, PLAIN, [NAME, file], NIL, NIL, base64, 2516, 33, NIL, [attachment,
[FILENAME, file]]],
[TEXT, PLAIN, [NAME, file], NIL, NIL, base64, 2516, 33, NIL, [attachment,
[FILENAME, file]]],
[TEXT, PLAIN, [NAME, file], NIL, NIL, base64, 2516, 33, NIL, [attachment,
[FILENAME, file]]],
[TEXT, PLAIN, [NAME, file], NIL, NIL, base64, 2516, 33, NIL, [attachment,
[FILENAME, file]]],
[TEXT, PLAIN, [NAME, file], NIL, NIL, base64, 2516, 33, NIL, [attachment,
[FILENAME, file]]],
MIXED,
[BOUNDARY, ----=_Part_0_31379709.1179480493454],
NIL,
NIL
]

Post by Chris Burdess

Post by remi__
As you suggested, here is the answer of themail server I use.
* 23 FETCH (BODYSTRUCTURE (("TEXT" "PLAIN" ("CHARSET"
"windows-1252") NIL
NIL "7bit" 9 1)("TEXT" "PLAIN" ("NAME" "file") NIL NIL "base64" 2516 33 NIL
("attachment" ("FILENAME" "file")))("TEXT" "PLAIN" ("NAME" "file") NIL NIL
"base64" 2516 33 NIL ("attachment" ("FILENAME" "file")))("TEXT" "PLAIN"
("NAME" "file") NIL NIL "base64" 2516 33 NIL ("attachment" ("FILENAME"
"file")))("TEXT" "PLAIN" ("NAME" "file") NIL NIL "base64" 2516 33 NIL
("attachment" ("FILENAME" "file")))("TEXT" "PLAIN" ("NAME" "file") NIL NIL
"base64" 2516 33 NIL ("attachment" ("FILENAME" "file")))
"MIXED" ("BOUNDARY"
"----=_Part_0_31379709.1179480493454") NIL NIL))
Content-Type: text/plain; name=file
Content-Transfer-Encoding: 33
javax.mail.MessagingException: Unknown encoding: 33
at javax.mail.internet.MimeUtility.encode(MimeUtility.java:240)
at javax.mail.internet.MimeBodyPart.writeTo(MimeBodyPart.java:804)
at TestGNUMail.displayMail(TestGNUMail.java:61)
at TestGNUMail.test(TestGNUMail.java:45)
at TestGNUMail.main(TestGNUMail.java:33)
Content-Id: base64
Content-Description: 2516
It seems that the "2516 33" pattern is recurrent in the answer. Is that
supposed to be so? I have no knowledge in the IMAP protocol.

The response looks mostly normal. It appears to have a content-
disposition in the extension where a content-language is expected,
but that certainly shouldn't break anything and would not cause the
content-id, content-description, and content-transfer-encoding to
have the wrong values.
Maybe the parsing of the IMAP structure is broken at a lower level.
If you have access to a debugging environment, could you post the
value of "list" in the IMAPMessage.parseMultipart method during
execution of your program?
_______________________________________________
Classpathx-javamail mailing list
http://lists.gnu.org/mailman/listinfo/classpathx-javamail

[Chris Burdess]

Post by remi__
This is the value of "list" just after the call to parseMultipart
(List list,
Part parent, InternetHeaders parentHeaders,
String baseSection)
[
[TEXT, PLAIN, [CHARSET, windows-1252], NIL, NIL, 7bit, 9, 1],
[TEXT, PLAIN, [NAME, file], NIL, NIL, base64, 2516, 33, NIL,
[attachment,
[FILENAME, file]]],
[TEXT, PLAIN, [NAME, file], NIL, NIL, base64, 2516, 33, NIL,
[attachment,
[FILENAME, file]]],
[TEXT, PLAIN, [NAME, file], NIL, NIL, base64, 2516, 33, NIL,
[attachment,
[FILENAME, file]]],
[TEXT, PLAIN, [NAME, file], NIL, NIL, base64, 2516, 33, NIL,
[attachment,
[FILENAME, file]]],
[TEXT, PLAIN, [NAME, file], NIL, NIL, base64, 2516, 33, NIL,
[attachment,
[FILENAME, file]]],
MIXED,
[BOUNDARY, ----=_Part_0_31379709.1179480493454],
NIL,
NIL
]

OK, thanks. Again, this all looks depressingly normal and doesn't
explain how you come to have your problem. I'm not sure what avenues
to suggest.

[remi__]

Here is an "avenue":

I ve been debuggin the parseMultiPart and parseBodyPart methods of
IMAPMessage.
Here is what I see in parseBodyPart :

list is [TEXT, PLAIN, [NAME, file], NIL, NIL, base64, 2516, 33, NIL,
[attachment, [FILENAME, file]]]
its size is 10

String encoding = parseAtom(list.get(BS_ENCODING + extensionCount));

with
BS_ENCODING = 5 (it is a constant)
and
extensionCount = list.size() - 8 = 2

so encoding is the 8th value of the list, which is 33

I would have expected extensionCount to be 0 so that we get the 6th value of
the list...

What I really don t get is what that does not happen when you execute it...

Post by Chris Burdess

Post by remi__
This is the value of "list" just after the call to parseMultipart
(List list,
Part parent, InternetHeaders parentHeaders,
String baseSection)
[
[TEXT, PLAIN, [CHARSET, windows-1252], NIL, NIL, 7bit, 9, 1],
[TEXT, PLAIN, [NAME, file], NIL, NIL, base64, 2516, 33, NIL,
[attachment,
[FILENAME, file]]],
[TEXT, PLAIN, [NAME, file], NIL, NIL, base64, 2516, 33, NIL,
[attachment,
[FILENAME, file]]],
[TEXT, PLAIN, [NAME, file], NIL, NIL, base64, 2516, 33, NIL,
[attachment,
[FILENAME, file]]],
[TEXT, PLAIN, [NAME, file], NIL, NIL, base64, 2516, 33, NIL,
[attachment,
[FILENAME, file]]],
[TEXT, PLAIN, [NAME, file], NIL, NIL, base64, 2516, 33, NIL,
[attachment,
[FILENAME, file]]],
MIXED,
[BOUNDARY, ----=_Part_0_31379709.1179480493454],
NIL,
NIL
]

OK, thanks. Again, this all looks depressingly normal and doesn't
explain how you come to have your problem. I'm not sure what avenues
to suggest.
_______________________________________________
Classpathx-javamail mailing list
http://lists.gnu.org/mailman/listinfo/classpathx-javamail

[Chris Burdess]

Post by remi__
I ve been debuggin the parseMultiPart and parseBodyPart methods of
IMAPMessage.
list is [TEXT, PLAIN, [NAME, file], NIL, NIL, base64, 2516, 33, NIL,
[attachment, [FILENAME, file]]]
its size is 10
String encoding = parseAtom(list.get(BS_ENCODING + extensionCount));
with
BS_ENCODING = 5 (it is a constant)
and
extensionCount = list.size() - 8 = 2
so encoding is the 8th value of the list, which is 33
I would have expected extensionCount to be 0 so that we get the 6th value of
the list...
What I really don t get is what that does not happen when you
execute it...

You must be working from version 1.1 or earlier of gnumail. You need
to update to the current version (1.1.2).

[remi__]

The update to 1.1.2 solved my problem.
Let's hope now that JOnAS switches to that version in the upcoming releases
!

Thanks for your support,

Rémi

Post by Chris Burdess

Post by remi__
I ve been debuggin the parseMultiPart and parseBodyPart methods of
IMAPMessage.
list is [TEXT, PLAIN, [NAME, file], NIL, NIL, base64, 2516, 33, NIL,
[attachment, [FILENAME, file]]]
its size is 10
String encoding = parseAtom(list.get(BS_ENCODING + extensionCount));
with
BS_ENCODING = 5 (it is a constant)
and
extensionCount = list.size() - 8 = 2
so encoding is the 8th value of the list, which is 33
I would have expected extensionCount to be 0 so that we get the 6th value of
the list...
What I really don t get is what that does not happen when you
execute it...

You must be working from version 1.1 or earlier of gnumail. You need
to update to the current version (1.1.2).
_______________________________________________
Classpathx-javamail mailing list
http://lists.gnu.org/mailman/listinfo/classpathx-javamail