Support Forum

Advanced Search
Forum Scope


Match



Forum Options



Minimum search word length is 3 characters - maximum search word length is 84 characters
plugins-topic
The Post-By-Email premium plugin is either not capturing or just not using html email content when making posts from email
Avatar
Ed Johnsen
Rookie
Free Members
sp_UserOfflineSmall Offline
Aug 30, 2019 - 3:57 pm

Hello,

After purchasing the $99 premium plugin bundle, I installed the HTML-Email plugin, the Post-By-Email plugin, and the Subscriptions plugin.

I am able to send nice html emails from the forum to email clients, but, when emails are sent from email clients to the forum, the html parts of the email never make it to the forum.

This makes is nearly impossible to read posts made via email that include quoted text from other emails and posts.

 

Although the sp-email-post-imap.php file appears to check for the html parts of an email, html parts are either never found, or always overwritten by plain-text parts before the contents of an email are added to a forum topic as a post.

For example, here is what shows up when trying to forward an html-email to the forum from any of the email clients I have tried:

> sent from post to email. > *Bold* > *Italic* > *Underline* > > Blockquote > > Is it easier to see the differentiation now, with the html-email premium plugin enabled?

The ">" characters seem to represent quoted text.

Newlines in quoted text are removed... this makes it nearly impossible to read forum contents posted by email.

HTML bold, italic, and underline tags are all replaced with *.

Blockquote is apparently the same thing as a quote in an email; we see ">>" when a blockquote exists in quoted text.

 

I believe that this is a bug; please let me know if it will be fixed, if it isn't a bug, or if there is some workaround.

Thanks for any assistance,

Ed

 

- Side note:
The sp-email-post-pop.php does not appear to include any code that would look for the html parts of an incoming email. (Worse yet, I've never seen simplepress successfully connect to a mail server via pop3, only imap).

Avatar
SP Community Support
SP Moderator
sp_UserOfflineSmall Offline
Aug 30, 2019 - 5:23 pm

Hi:

Only text is imported from incoming emails.  Importing random html elements will break most forums (and likely introduce some security issues.)

Thanks.

Avatar
Ed Johnsen
Rookie
Free Members
sp_UserOfflineSmall Offline
Aug 30, 2019 - 6:15 pm

Thank you for the reply.

Does SimplePress provide hooks or other ways to readily extend the functionality of post-by-email so that I can let my select group of privileged-to-post-to-the-forum users post emails that are readable when they include inline-responses?

 

It seems that you've already done some of the work required to support html-from-email in your sp-email-post-imap.php file; on these lines:

$body_structure = sp_pbe_body_structure_for_html_part($body_structure);

function sp_pbe_is_mime_subtype_html($a_part) {
	return(sp_pbe_is_mime_subtype_specified($a_part) && sp_pbe_does_mime_subtype_match_given_string($a_part, "HTML"));
}

# tests the key parameters of the part and returns true if it looks like a properly formed HTML part.
function sp_pbe_is_part_a_valid_html_part($a_part) {
	return(sp_pbe_is_primary_body_type_text($a_part) && sp_pbe_is_mime_subtype_html($a_part));
}

# internally used by sp_pbe_body_structure_for_html_part().
function sp_pbe_get_a_default_part($a_structure) {
	$result = $a_structure; # if for some reason we don't find a better part to assign, just use this top-level structure.
	if(is_array($a_structure->parts) && count($a_structure->parts) >= 1) {
		$result = $a_structure->parts[0];
	}
	return $result;
}

# works through the given parts in the structure to find
# a well-formed HTML part. Returns that part if it finds
# it. Returns null otherwise.
# internally used by sp_pbe_body_structure_for_html_part().
function sp_pbe_scan_for_html_part($the_parts_array) {
	$result = null;
	foreach($the_parts_array as $this_part) {
		if(sp_pbe_is_part_a_valid_html_part($this_part)) {
			$result = $this_part; # found our matching part
			break;
		}
	}
	return $result;
}

# works through the given parts in the structure to find
# a well-formed PLAIN part. Returns that part if it finds
# it. Returns null otherwise.
# internally used by sp_pbe_body_structure_for_html_part().
function sp_pbe_scan_for_plain_part($the_parts_array) {
	$result = null;
	foreach($the_parts_array as $this_part) {
		if(sp_pbe_is_part_a_valid_plain_part($this_part)) {
			return $this_part; # found our matching part
			break;
		}
	}
	return $result;
}

# gets the structure of the most appropriate part of a multipart
# message that we can find. We use this 'most appropriate part'
# to identify how the raw content of the message has been encoded.
function sp_pbe_body_structure_for_html_part($a_structure) {
	$parts = $a_structure->parts;

	$result = sp_pbe_scan_for_html_part($parts);
	if(is_null($result)) {
		# if we didn't find an HTML part, now fall back to looking for a PLAIN part
		$result = sp_pbe_scan_for_plain_part($parts);
		if(is_null($result)) {
			# still didn't find a matching part, so fall back to our best default part option
			$result = sp_pbe_get_a_default_part($a_structure);
		}
	}

	return $result;
}






Further, you already allow "arbitrary html" to be posted to the forum directly, and already have privileging structures in place that 
strip, say, iframes from forum posts by users not allowed to post iframes; it seems arbitrary to block html from an 
email source when all other posting options support html... via paid plugin.

Am I wrong in assuming that html from an email would only cause problems if it was poorly formed (eg, no closing tag for an opening tag)? 
To this end, php libraries like simpledom could help by completing un-closed tags and otherwise allowing you guys to 
loop through valid tags for html content.
Avatar
SP Community Support
SP Moderator
sp_UserOfflineSmall Offline
Sep 10, 2019 - 4:02 am

Hi:

HTML allowed on our forum vs HTML allowed in emails are two different things. 

HTML allowed on our forums can be controlled and filtered at the time the data is entered.  We can control what is allowed and what is not in real-time. 

With email, you never know what you're going to get because you're getting it after-the-fact and based on someone else's rules and whims. 

Between the various email clients that format their messages differently and the numerous servers an email might pass through before it gets to your inbox, there is zero control over what you're actually getting.  You can easily spend a year writing code for fiddling with stripping and formatting emailed html.  Even the biggest companies haven't gotten it right - though, of course, they do it better than most.  We've done it for other large software projects before so we know the issues - not a week goes by where we didn't see a new twist on what email clients are sending over and having to write new code to handle it. 

The most reliable thing you can do is extract the text which is what we do for Simple:Press.  The code you posted does make a very very basic attempt at extracting the html but unless it comes in absolutely perfectly, you're going to get the text portion only. 

Thanks.

Forum Timezone: Europe/Stockholm
Most Users Ever Online: 1170
Currently Online:
Guest(s) 1
Currently Browsing this Page:
1 Guest(s)
Top Posters:
Mr Papa: 19448
Ike: 2086
Brandon: 864
kvr28: 804
jim: 650
FidoSysop: 577
Conrad_Farlow: 531
fiddlerman: 358
Stefano Prete: 325
Member Stats:
Guest Posters: 620
Members: 17365
Moderators: 0
Admins: 4
Forum Stats:
Groups: 7
Forums: 17
Topics: 10128
Posts: 79626