Support Forum

Tags System

MP Mr Papa
Mr Papa
Member
Member since Dec 2006
19,448 posts
Mar 24, 2009 · 8:10 pm · Edited

First, some surprise – for me at least…  went looking at a whole bunch of forum sw the last couple days to see how they handled tags and how they might compare to what I was thinking for an implementation.  To my surprise, the only one I found doing tags was BBPress.  There must be others and I must have missed them or its a feature folks arent using…  Still, I think the power of a tagging system is there…  guess we can be trendsetters…

First, the easy part… I plan to allow tagging to be enabled or disabled at the forum level.

So next question becomes where do we allow tagging?  at the topic level or the post level?  My first instinct was at the topic level.  So, whoever creates the topics, sets the tags for that topic.  But, I could also see allowing the adding of tags at the post level.  The mechnaism of adding tags wont really care.  If we do tags at the post level, do we show the tags at the topic level or at a post by post level?  Certainly possible to show summary tags at the topic level, but allow users to add new tags at the post level, but of course then we would need to verify that any added tags dont already exist at the topic level.  if the tags were kept at the post level, then no worry about duplicates.  I can certainly see some benefit to allowing tagging at the post level, but I suspect 90 or 95% of the time, the topic tags (ie the first post in topic) will be sufficient.  Take WP for example, you tag the post and others dont tag the comments.  BBPress follows the same idea and tags at the topic level with the first post.  So, I think we tag at the topic level.

How are tags entered?  Tags would be entered into a input field as a comment separated list of key words of phrases.  Would have to do some checking/stripping for strange characters/entities/etc.

How are tags stored in the db?  I envision creating a new table called sftags.  The columns in this new table woud be tag_id, topic_id, tag and tag_slug.  Each tag entered for a new topic would get its own row in this table.   The tag slug would be used for potential methods of accessing topics by tags (ie url based).

Now, this may not be the most efficient db usage and could grow to be very large.  If we wanted to save some db size at the expense of complexity and perhaps queries, an alternative method would be to use two tables.  One would be sftags which would be comprised of tag_id, tag and tag slug.  The other table would be sftagsmeta.  This would be comprised of meta_id and topic_id.  When a topic was created, for each tag, if it was a new tag based on looking in sftags table, it would be added to the sftags table and an entry for the topic would be added to sftags meta that pointed to the tag in sftags.  If the tag already existed in sftags, then simple a new entry in sftagsmeta would be made pointing to the existing tag in sftags. 

Now, not entirely sure of the benefit of the two table system but it just seems like this provides more growth options, perhaps like adding another column to the sftags table for topic_count – how many topics have this tag.  Might be a way to easily have a listing of most popular tags.  there is probably other benefits and other database options.  I guess my current choice would be to go with the two table option.

So, where are tags entered?  It doesn really matter whether its topics, as I suggest, or all posts.  Thinking of something like the mockup below:

So then, where are tags displayed? 

Do we display the tags on the forum view?  I would think so. We could put a list of the tags in a table underneath the forum title, but there is already a bunch of stuff in that field.  We could add a new column, but that would likely look weird for a bunch of tags.  Another option would be another, short (ie not tall vertically) row that spans the entire length of the topic entry (except for the first, topic icon column?).  I would try both the under topic title and new row displays to see which one I like better.  I would vote against the additional column.

Do we display the tags on the topic view?  This would be a given I believe.  The location of the display is even more tricky than the forum view.  The most obvious places would be at the top  or the bottom of the page.  I would suggest above or below the sffooter regions (page links).  This tag display seems like something that would be nice in the sidebar, but thats probably not a good thing to do and would confuse folks on how to do it.

So, now that we have tags, what do we do with them?  A couple of ideas, certainly a lot more possibilities

  • search by tag
  • most popular tags
  • tag clouds (forum or topic)
  • related/similar topics

thoughts?

39 Answers

New Answer
YS Yellow Swordfish
Yellow Swordfish
Member
Member since Nov 2008
16,437 posts
Mar 25, 2009 · 3:08 am

Statement of Intent:
The first and obvious question that will be asked on day one of release is – 'can we share forum and blog tags?' In some respects it would be great to share but if we did that then users would also expect forum topics to appear in blog tag 'views', clouds, lists and searches and to be honest that is a nightmare – although perhaps one day if someone feels up to rewriting the WP search routines….! So I – with you – vote for keeping this independent of the blog and making big, bold statements to that effect. (Do we know if bbPress shares the blog tags?)

Tag Administration:
I agree turning tags on/off at forum level seems like a sound approach. Removes the need to have a global 'switch'. However, I do think this should also be a new permission.

We should also ponder whether some forum admins would prefer to pre-create a whole bunch of tags for their members to use and disallow members creating their own.

Tag Entry:
This really has to be at topic level when a new topic is created. I would also like to see a list of current tags (multi-select list) that can be selected for use as well as entering in a new one if allowed to do so. It may also be worth considering a global setting defining how many tags can be attached to a topic to stop members getting carried away!

Post Entry:
I suggest a new admin tool icon at both topic and post level that would allow an Admin (or Moderator with permission) to change the tags. Add/remove and edit for spelling mistakes etc. The problem with end-user defined tags will always be redundancy. One will enter a tag of say 'editor', another 'editors' another 'editing'. Somehow the admin needs the tools to clean such anomalies up.

Tag Display – Forum View:
I agree with your notion of a separate row but we should take the opportunity of doing this with re-designing what we currently have. There are potentially status icons, rating appregate, topic status and now tags. Rather than just add a row let's actually try and redesign this for the best approach with narrow columns in mind.

Tag Display – Topic View:
I agree – at the top and optionally bottom. Same applies though. Let's take the opportunity to redesign what we have now and do it properly.

Other Items:
Adding this to the search bar (in a similar way to topic status searching) is absolutely essential. And yes – there will be a need for template tags. We should also review current tags to see if any would benefit from having tags added.

Database:
Assuming we do not want to use the built in taxonomy system (which would work) then this must be handled by two tables. Let's break with WP tradition and make these properly relational (although the taxonomy system does meet those requirements). So yes – an 'sftags' table with:

tag_id (primary/ndexed/auto-increment)
tag_name
(tag_slug – not sure we would ever want this but no harm).

Adding a topic_count into this table breaks relational rules but it would be a boost to performance and I strongly urge it to be incorporated.

'sftagmeta' is probably as good a name as any for the other table but I would be tempted to use the two key names: 'topic_id' and 'tag_id'. We need to investigate the best indexing approach on this table which will be crucial for performance. It might be that a compound index would be better but we can check this.

Queries:
From my point of view the most critical part of all of this will come down to the queries for forum and topic views. It needs to be very carefully crafted into the current query model so as not to trigger extra queries and this should, I believe, be possible with care.

Potential Problems:
I think the main problem I see comes down to letting members loose defining tags all over the place – creating duplicates, typos, spelling errors, problems with case etc. I, as a forum admin, want to be able to keep a tag list clean and accurate and for it to truly reflect the topic it is attached to. I want a tool that will allow me to do this and that will need some smart backend routines.

MP Mr Papa
Mr Papa
Member
Member since Dec 2006
19,448 posts
Mar 25, 2009 · 12:03 pm

Statement of Intent:

Agree wholeheartedly about not mixing blog tags and spf tags… thought about it during my study, but the negatives far out weight the positivies, I believe.

Tag Administration:

I had considered the permission but came down against it.  I am not sure I see the need for it as if I want tags, why would I want to limit who could add tags?  just doesnt make sense to me.  but then again, the way other folks have implemented and used the forums dont make sense either.  You are probably right, folks will ask for it, but I just dont get it…  will put it back on the table for consideration…  easy enough to add after the tags are working…

I guess I see even less reason for having a predefined set of tags.  That seems like topic status territory – not tags.  Really dont want to do the predefined set of tags. 

I am also envisioning and planning, though didnt mention it, a backend admin page for adminsitering tags.  You will be able to view, rename, delete, etc tags from here.  I have extensively used the best and most popular wp tagging plugin (simple tags) and plan to mimic some of their stuff.  Managing which tags go to which topic wont be too easy in the backend short of having a complete listing of topics which doesnt sound pleasant or efficient.  can take a look at.

I must admit, I had not thought about adding a new admin icon tool, but at first glance, it seems like an idea that makes sense.  This would also be the best spot for managing the tags on a given topic.  However, it might get pretty complex pretty quickly. Will have a play with it.

Tag Entry:

Not sure I see a good reason to limit the number of tags, but not hard over on that one.  For true tagging, a limit doesnt really make sense.  Also the UI gets tricky because I also believe the input field to be the proper format for entering tags and not a dropdown list.  Limiting tags I guess would mean chopping off the last ones.   The dropdown select list will get super unwieldly real fast.  As somone who extensively uses tags on my blog posts, I have hundred or thousands.  A dropdown simply would not work.  One needs to use the best words for describing the post. tags are not categories or keywords. think of them as related words that describe the post.  I knew you would want some sort of existing tag interface UI and want to do what I have for my post tags which is have the tagging system suggest a list of potential tags based on the post content (works pretty darn good).  The suggested tags open in a hidden area below tag input field when a suggest button is clicked.  Each tag listed there is clickable and would insert it into the input field.  Really slick but didnt want to promise it since it will be something new for me.

Tag View Forum:

open to suggestions here, but my new row suggestion would have spanned all the existing cols (except for maybe first) and been dedicated to tags since they could use the entire width if there are lots (hope not that wide, but you never know).  I would make it a small height row and perhaps a smaller font.  Not sure a redesign is needed, but certainly willing to consider.

Tag View Topic:

Same applies for redesign ideas.  I had been thinking a new block.

Database:

Lol, had not even considered what WP does – guess I should look but doubt its what I would want to do, but who knows…

Think we are in total agreement on the database structure.   I did mean to include the tag_id in the sftagmeta table, just forgot it.  I was also recommending the two table option also.  You might have to help me with some of the mysql terminology you used.  I will post tonight what I was going to do for creating the tables.  I have no idea what a compound index is.  I had planned on indexing the tag name, but certainly open to suggestions.  So let me perhaps research compound indexes tonight and post my suggestion.  You certainly know more about mysql than I…

Queries:

Afraid, and I havent not yet looked at this, but I would doubt this couldnt be any additional queries.  It seems to access additional tables, you would at least have to do a join… but definitely agree on minimizing any impact.  you spent lots ot time and effort improving this for 4.1 and dont want to onduly increase it for tags.

Search:

Will likely need some help here but perhaps time to learn how our searching works…

YS Yellow Swordfish
Yellow Swordfish
Member
Member since Nov 2008
16,437 posts
Mar 25, 2009 · 2:17 pm

Actually – the search is probably the easiest bit of the lot to be honest! Smile

'Compound Index': might not be what MySQL cals it – I have no idea – but it means an index of more than one column literally combined. Probably not worth doing but I will check with ne of my ex-colleagues. It can be added later if it helps.

Can I just get one thing straight? – example: I tag Topic A with the word 'chocolate'. It becomes tag id 100. Later YOU tag topic B also with the term 'chocolate'. I am hoping that Topic B is going to be linked to tag id 100 also. In other words, it is important not to spawn a new tag record when one exists. And that is why I mentioned the horrible problem with things like case. Is 'chocolate' the same tag as 'Chocolate'? (Actually as a matter of course I would be tempted to filter all tags to lower case before saving them).

On the issue of re-design. it may be tha they don't need it but these areas are starting to look quite cluttered and I think we really do need to review them and adjust where necessary.

Oh and yes – havng new admn tool icons would be a good thing. And expected really I think.

MP Mr Papa
Mr Papa
Member
Member since Dec 2006
19,448 posts
Mar 25, 2009 · 6:19 pm

Can I just get one thing straight? – example: I tag Topic A with the word 'chocolate'. It becomes tag id 100. Later YOU tag topic B also with the term 'chocolate'. I am hoping that Topic B is going to be linked to tag id 100 also. In other words, it is important not to spawn a new tag record when one exists. And that is why I mentioned the horrible problem with things like case. Is 'chocolate' the same tag as 'Chocolate'? (Actually as a matter of course I would be tempted to filter all tags to lower case before saving them).

Absolutely on the first part – I think…Wink

I was not planning on linking Topic B to the sftag id…  Thats what the second table sftagmeta was for…  There would be a new entry in there that pointed to the topic id and to the already existing entry in sftag…  I believe thats what you mean, but just making sure…

As to your case, I was/am considering two options.  One would be to filter the tags to lower case.  Not that good if the tag is a name or proper noun (hope thats the right English), but livable.  Was also considering using the tag slug for comparisons vs the tag name (as well as any url references).  the tag slug will get filtered, escaped, cleaned up, etc like any other tag and be pretty darn clean.  That way the display name can be however “fancy” wanted (albeit restricted to the first enry that creates it).  One problem with this method then is that tags cannot have a dash in them, but thats quite common for tag implementations.

Not really looking forward to coding up that admin tool icons part, but the more I think about it the more it makes sense for managing tags at the topic level. Can still consider listing all the topics in the backend, but that would still be painful to manage tags at the topic level.

MP Mr Papa
Mr Papa
Member
Member since Dec 2006
19,448 posts
Mar 25, 2009 · 6:38 pm

This is the mysql I was thinking about for creating the new tables…

For the new TAGS table…

                $sql = “
                    CREATE TABLE IF NOT EXISTS “.SFTAGS.” (
                        tag_id bigint(20) NOT NULL auto_increment,
                        tag_name varchar(50) default NULL,
                        tag_slug varchar(50) default NULL,
                        tag_count bigint(20) default '0',
                        PRIMARY KEY  (tag_id),
                        FULLTEXT KEY tag_name (tag_name)
                    ) ENGINE=MyISAM “.sf_charset().”;”;
                $wpdb->query($sql);

and for the new TAGSMETA table…

                $sql = “
                    CREATE TABLE IF NOT EXISTS “.SFTAGMETA.” (
                        meta_id bigint(20) NOT NULL auto_increment,
                        tag_id bigint(20) default '0',
                        topic_id bigint(20) default '0',
                        PRIMARY KEY  (meta_id),
                        KEY tag_idx (tag_id),
                        KEY topic_idx (topic_id)
                    ) ENGINE=MyISAM “.sf_charset().”;”;
                $wpdb->query($sql);

Comments suggestions and improvments are certainly welcome…

MP Mr Papa
Mr Papa
Member
Member since Dec 2006
19,448 posts
Mar 26, 2009 · 12:14 pm

is that what you were referring to as a Compound Index?

MP Mr Papa
Mr Papa
Member
Member since Dec 2006
19,448 posts
Mar 27, 2009 · 10:27 am

any advice as to the filtering of the tag name and tag slug?  worried about other languages.  was thinking of pretty much using the topic name filtering we do now on the tag name and the topic slug filtering we do on tag slug.

Still planning to check for existing tags by the tag slug vs the tag name.

YS Yellow Swordfish
Yellow Swordfish
Member
Member since Nov 2008
16,437 posts
Mar 27, 2009 · 11:46 am

use the built in create slug routine although you wil probably need to add a new 'type'. The code is obvious.

Wealso have a couple of outstanding upgrades to this routine to help non-English langauges so the tag slugs will automatically benefit from those as well.

MP Mr Papa
Mr Papa
Member
Member since Dec 2006
19,448 posts
Mar 27, 2009 · 12:02 pm

I used the slug routine, but didnt update it…  I want to skip all that duplicate stuff as I have to do something different…  working fine…

not sure if an escape if enough on the name, which is what do on topic title, but thats what I am doing…  adding tags working great (manual only)…  thinking about checking it in before I do the display or back end stuff..

MP Mr Papa
Mr Papa
Member
Member since Dec 2006
19,448 posts
Mar 27, 2009 · 11:55 pm

okay, a few basic things have been committed to trunk for 4.1 regarding tags…

  • users can enable/disable tags on a forum by forum basis
  • users can limit the number of tags entered for a topic
  • forum users can enter tags manually or can click on a suggest link from one of three places (current spf tags, yahoo or tagthis.net) based on post content – works for all editors
  • topic tags are displayed in forum views

as usual, any and all comments are welcome on the current state…

so, whats left?

  • styling of whats there now (including potential forum view redesign)
  • tags display at the topic level
  • search by tags
  • consider making the topic tags display (each tag) a link to search by that tag
  • back end admininstration to mass edit, rename and delete tags
  • optimize the queries at the forum level (currently one query per topic)
  • admin icon tools option to edit tags at the topic level
  • template tags
  • give credit to simple-tags guys for copious reuse of their code

and so, what have I missed?

MP Mr Papa
Mr Papa
Member
Member since Dec 2006
19,448 posts
Mar 29, 2009 · 5:22 pm

an update on the tags situation…

  • users can enable/disable tags on a forum by forum basis
  • users can limit the number of tags entered for a topic
  • forum users can enter tags manually or can click on a suggest link from one of three places (current spf tags, yahoo or tagthis.net) based on post content – works for all editors
  • topic tags are displayed in forum views
  • topic tags are displayed in topic views
  • initial cut at back end admininstration to mass edit, rename and delete tags as well as see what topics they are tied too
  • Tags housekeeping routine in the admin
  • Queries optimized for forum and topic view such that only 1 query was added per view

as usual, any and all comments are welcome on the current state…

so, whats left?

  • styling of whats there now (including potential forum/topic view redesign)
  • search by tags
  • consider making the topic tags display (each tag) a link to search by that tag
  • admin icon tools option to edit tags at the topic level
  • template tags
  • give credit to simple-tags guys for some reuse of their code and UI design
  • popup help for manage tags panel (mass edit is done)
  • testing, testing, testing

and so, what have I missed?

YS Yellow Swordfish
Yellow Swordfish
Member
Member since Nov 2008
16,437 posts
Mar 30, 2009 · 2:19 am

The only thing I think is slightly odd (but not much!) is the 'suggest tags' feature. The one place it doesn't suggest tags from is the current tag list within the forum itself…