vanderwal.net

Wikipedia Folksonomy is a Mess with Collaborative Misunderstanding

There are days I really wish there were a test for people to get the ability to have access to edit Wikipedia. In the past month I have been asked over and over about folksonomy being a collaborative tagging, which makes no sense to people. It is tough to makes sense of the relationship to folksonomy as there is no relationship, as folksonomy is a collective tagging practice not collaborative. People depend on Wikipedia, which is the case of folksonomy they should not.

Collaboration and collective efforts are often confused by those not familiar with both terms, but they are not similar and they are two distinctly different efforts. Collaboration is people working together (often with a common goal) to build one thing (think wiki page with one understanding). Collective efforts are the aggregation of people's individual efforts, sometimes in the same service, but do not have common goal or common effort (del.icio.us page for a URL is the collective understanding of individuals tagging of that page for their own use.

Call to Fix Wikipedia entry on Folksonomy, Again

Can we get the mis-understood "collaborative tagging" out of the folksonomy page, since the descriptions around collective tagging are clearly a mis-understanding of the term collaborative, when they meant collective. There are collaborative tagging efforts in content management tools, where they are working toward one single on a limited understanding of terms, but that is not what has been described.

You ask why I do not change the page Wikipedia Folksonomy page. I did it a long time ago to fix the understanding and a friend pointed out that the edit was mine and I got irate e-mail. A since that time Wikipedia has modified its policies to improve the virtrol in Wikipedia.

A Stale State of Tagging?

David Weinberger posted a comment about Tagging like it was 2002, which quotes Matt Mower discussing the state of tagging. I mostly agree, but not completely. In the consumer space thing have been stagnant for a while, but in the enterprise space there is some good forward movement and some innovation taking place. But, let me break down a bit of what has gone on in the consumer space.

History of Tagging

The history of tagging in the consumer space is a much deeper and older topic than most have thought. One of the first consumer products to include tagging or annotations was the Lotus Magellan product, which appeared in 1988 and allowed annotations of documents and objects on one's hard drive to ease finding and refinding the them (it was a full text search which was remarkably fast for its day). By the mid-90s Compuserve had tagging for objects uploaded into its forum libraries. In 2001 Bitzi allowed tagging of any media what had a URL.

The down side of this tagging was the it did not capture identity and assuming every person uses words (tag terms) in the same manner is a quick trip to the tag dump where tags are not fully useful. In 2003 Joshua Schacter showed the way with del.icio.us that not only allowed identity, upon which we can disambiguate, but it also had a set object in common with all those identities tagging it. The common object being annotated allows for a beginning point to discern similarity of identityĵs tag terms. Part of this has been driven on Joshua's focus on the person consuming the content and allowing a means for that consumer to get back to their information and objects of interest. (It is around this concept that folksonomy was coined to separate it from the content publisher tagging and non-identity related tagging.) This picked up on the tagging for one's self that was in Lotus Magellan and brings it forward to the web.

Valuable Tagging

It was in del.icio.us that we saw tagging that really did not work well in the past begin to become valuable as the clarity in tag terms that was missing in most all other tagging systems was corrected for in the use of a common object being tagged and the identity of the tagger. This set the foundation for some great things to happen, but have great things happened?

Tagging Future Promise

Del.icio.us set many of out minds a flutter with insight into the dreams of the capability of tagging having a good foothold with proper structure under them. A brilliant next step was made by RawSugar (now gone) to use this structure to make ease of disambiguating the tag terms (by appleseed did you mean: Johnny Appleseed, appleseeds for gardening/farming, the appleseed in the fruit apple, or appleseed the anime movie?). RawSugar was a wee bit before its time as it is a tool that is needed after there tagging (particularly folksonomy related tagging systems) start scaling. It is a tool that many in enterprise are beginning to seek to help find clarity and greater value in their internal tagging systems they built 12 to 18 months ago or longer. Unfortunately, the venture capitalists did not have the vision that the creators of RawSugar did nor the patience needed for the market to catch-up to the need in a more mature market and they pulled the plug on the development of RawSugar to put the technology to use for another purpose (ironically as the market they needed was just easing into maturity).

The del.icio.us movement drove blog tags, laid out by Technorati. This mirrored the previous methods of publisher tagging, which is most often better served from set categories that usually are derived from a taxonomy or simple set (small or large) of controlled vocabulary terms. Part of the problem inherent in publisher tags and categories is that they are difficult to use outside of their own domain (however wide their domain is intended - a specific site or cross-sites of a publisher). Using tags from one blog to another blog has problems for the same reason that Bitzi and all other publisher tags have and had problems, they are missing identity of the tagger AND a clear common object being tagged. Publisher tags can work well as categories for aggregating similar content within a site or set of commonly published sites where a tag definition has been set (but that really makes them set categories) and used consistently. Using Technorati tag search most often surfaces this problem quickly with many variation of tag use surfacing or tag terms being used to attract traffic for non-related content (Technorati's keyword search is less problematic as it relies on the terms being used in context in the content - unfortunately the two searches have been tied together making search really messy at the moment). There is need for an improved tool that could take the blog tags and marry them to the linked items in the content (if that is what is being talked about - discerning predicate in blog tags is not clear yet).

Current Tools that Advanced

As of a year ago there were more than 140 social bookmarking tools in the consumer space, but there was little advancement. But, there are a few services that have innovated and brought new and valuable features to market in tagging. As mentioned recently Ma.gnolia has done a really good job of taking the next steps with social interaction in social bookmarking. Clipmarks pioneered the sub-page tagging and annotation in the consumer tagging space and has a really valuable resource in that tool. ConnectBeam is doing some really good things in the enterprise space, mostly taking the next couple steps that Yahoo MyWeb2 should have taken and pairing it with enterprise search. Sadly, del.icio.us (according to comments in their discussion board) is under a slow rebuilding of the underlying framework (but many complaints from enterprise companies I have worked with and spoken indepth with complain del.icio.us continually blocks their access and they prefer not to use the service and are finding current solutions and options to be better for them).

A Long Way to Go

While there are examples that tagging services have moved forward, there is so much more room to advance and improve. As people's own collection of tagged pages and objects have grown the tools are needed to better refind them. This will require time search and time related viewing/scanning of items. The ability to use co-occurance of tag terms (what other tags were used on the object), with useful interfaces to view and scan the possibilities.

Portability and interoperability is extremely important for both the individual person and enterprise to aggregate, migrate, and search across their collections across services and devices (now that devices have tagging and have had for some time, as in Mac OS X Tiger and now Vista). Enterprises should also have the ability to move external tagged items in through their firewall and publish out as needed, mostly on an employee level. There is also desire to have B2B tagging with customers tagging items purchased so the invoicing can be in the customers terminology rather than the seller terminology.

One of the advances in personal tagging portability and interoperability can easily be seen when we tag on one device and move the object to a second device or service (parts of this are not quite available yet). Some people will take a photo on their mobile phone and add quick tags like "sset" and others to a photo of a sunset. They send that photo to a service or move it to their desktop (or laptop) and import the photo and the tag goes along with it. The application sees the "sset" and knows the photo was transfered from that person's mobile device and knows it is their short code for "sunset" and expands the tag to sunset accordingly. The person then adds some color attribute tags to the photo and moves the photo to their photo sharing service of choice with the tags appended.

The current tools and services need tools and functionality to heal some of the messiness. This includes stemming to align versions of the same word (e.g. tag, tags, tagging, bookmark, bookmarking). Tag with disambiguation in mind by offering co-occurrence options (e.g. appleseed and anime or johnny or gardening or apple). String matching to identify facets for time and date, names (from your address book), products, secret tag terms (to have them blocked from sharing), etc. (similar to Stikkit and GMail).

Monitoring Tools

Enterprise is what the next development steps really need to take off (these needs also apply to the power knowledge worker as well). The monitoring tools for tags from others and around objects (URLs) really need to fleshed out and come to market. The tag monitoring tools need to become granular based on identity and co-occurance so to more tightly filter content. The ability to monitor a URL and how it is tagged across various services is a really strong need (there are kludgy and manual means of doing this today) particularly for simple and efficient tools (respecting the tagging service processing and privacy).

Analysis Tools

Enterprise and power knowledge workers also are in need of some solid analysis tools. These tools should be able to identify others in a service that have similar interests and vocabulary, this helps to surface people that should be collaborating. It should also look at shifts in terminology and vocabulary so to identify terms to be added to a taxonomy, but also provide an easy step for adding current emergent terms to related older tagged items. Identify system use patterns.

Just the Tip

We are still at the tip of the usefulness of tagging and the tools really need to make some big leaps. The demands are there in the enterprise marketplace, some in the enterprise are aware of them and many more a getting to there everyday as the find the value real and ability to improve the worklife and workflow for their knowledge workers is great.

The people using the tools, including enterprise need to grasp what is possible beyond that is offered and start asking for it. We are back to where we were in 2003 when del.icio.us arrived on the scene, we need new and improved tools that understand what we need and provide usable tools for those solutions. We are developing tag islands and silos that desperately need interoperability and portability to get real value out of these stranded tag silos around or digital life.

Why Ma.gnolia is One of My Favorite Social Bookmarking Tools

After starting the Portable Social Network Group in Ma.gnolia yesterday I received a few e-mails and IMs regarding my choice. Most of the questions were why not just use tags and del.icio.us. After I posted my Ma.Del Tagging Bookmarklet post I have had a lot of questions about Ma.gnolia and my preference as well as people thought I was not a fan of it. I have been thinking I would blog about my usage, but given my work advising on social bookmarking and social web, I shy away talking about what I use as what I like is likely not what is going to be a good fit for others. But, my work is one of the reasons I want to talk about what I like using as nearly every customer of mine and many presentation attendees look at del.icio.us first (it kicked the door wide open with a tool that was light years ahead of all others), but it is not for everybody and there are many other options. Much of my work is with enterprise and organizations of various size, which del.icio.us is not right for them for privacy reasons. I still add to del.icio.us along with my favorite as there are many people that have subscribed to the at feed as they derive value from that subscription so I take the extra step to keep that feed as current.

Ma.gnolia Offers Great Features for Sociality

I have two favorite tools for my own personal social bookmarking reasons Ma.gnolia and Clipmarks (I don't think I have anything publicly shared in Clipmarks). First the later, I use Clipmarks primarily when I only want to bookmark a sub-page element out on the web, which are paragraphs, sentences, quotes, images, etc.

I moved to try Ma.gnolia again last Fall when something changed in del.icio.us search and the results were not returning things that were in del.icio.us. My trying Ma.gnolia, by importing all of my 2200 plus bookmarks not only allowed me to search and find things I wanted, but I quickly became a fan of their many social features. In the past year or less they have become more social in insanely helpful and kind ways. Not only does Ma.gnolia have groups that you can share bookmarks with but there is the ability to have discussions around the subject in those groups. Sharing with a group is insanely easy. Groups can be private if the manager wishes, which makes it a good test ground for businesses or other organizations to test the social bookmarking waters. I was not a huge fan of rating bookmarks as if I bookmarked something I am wanting to refind it, but in a more social context is has value for others to see the strength of my interest (normall 3 to 5 stars). One of my favorite social features is giving "thanks", which is not a trigger for social gaming like Digg, but is an interpersonal expression of appreciation that really makes Ma.gnolia a friendly and positive social environment.

Started with Beauty, but Now with Ease

Ma.gnolia started as a beautiful del.icio.us (it was not the first) and the beauty got in the way of usability for many. But, Ma.gnolia has kept the beautiful strains and added simple ease of use in a very Apple delightful moments sort of way. The thanks are a nice treat, but the latest interactions that provide non-disruptive ease of use to accomplish a task, without completely taking you away from your previous flow (freaking brilliant in my viewpoint - anything that preserves flow to accomplish a short task is a great step). Another killer feature is Ma.gnolia Roots, which is a bookmarklet that when clicked hovers a semi-transparent layer over the webpage to show information from Ma.gnolia about that page (who has linked to it, tags, annotations, etc.) and makes it really easy to bookmark that page from that screen. The API (including a replica of the del.icio.us API that nearly all services use as the standard), add-ons, Creative Commons license for your bookmarks, many bookmarklet options, and feed options. But, there are also the little things that are not usually seen or noticed, such as great URLs that can be easily parsed, all pages are properly marked up semantically, and Microformats are broadly and properly used throughout the site (nearly at every pivot).

Intelligently Designed

For me Ma.gnolia is not only a great site to look at, a great social bookmarking site that is really social (as well as polite and respectful of my wishes), but a great example for semantic web mark-up (including microformats). There is so much attention to detail in the page markup that for those of us that care it is amazingly beautiful. The visual layer can be optimized for more white space and detail or for much easier scrolling. The interactions, ease of use, and delightful moments that assist you rather than taking you out of your flow (workflow, taskflow, etc.) and make you ask why all applications and social sites are not this wonderful.

Ma.gnolia is not perfect as it needs some tools to better manage and bulk edit your own bookmarks. It could use a sort on search items (as well as narrow by date range). Search could use some RedBull at times. It could improve with filtering by using co-occurance of tag terms as well as for disambiguation.

Overall for me personally, Ma.gnolia is a tool I absolutely love. It took the basic social bookmarking idea in del.icio.us and really made it social. It has added features and functionality that are very helpful and well executed. It is an utter pleasure to use. I can not only share things easily and get the wonderful effects of social interaction, but I can refind things in my now 2,500 plus bookmarks rather easily.

Ma.gnolia Portable Social Networking Group Now Open

I have been talking with people about portable social networks for 18 months or more and initially blogged about it last November (2006) in my post Following Friends Across Walled Gardens. Recently the portable social network effort has flowed into Microformats: Social Network Portability. I have been following Brian Oberkirk's portable social network blog posts and we have had more than a few chats about this in the last few months. Finally it seems that some core geeks are on this quest as well, thanks to a gathering of minds at Foo Camp with Brad Fitzpatrick and David Recordon posting Thoughts on the Social Graph and the starting of the Social Network Portability Google Group. The oauth (more info on OpenAuth). Kevin Lawver has been rocking the real world with his Portable Social Networks at Mashup Camp discussion and example.

Ma.gnolia Portable Social Network Group

Tracking these small bits that are loosely joined needed a little more glue. To this end I started a ma.gnolia group for portable social networking to aggregate links. Already there is a good groups of people joining the group, which is promising. I have been critical of Ma.gnolia in the past, but they have iterated and built a social bookmarking site that has become my favorite social bookmarking service (Clipmarks is my second favorite when I need to just hold onto sub-page items). If you want to keep follow, keep track of the current site, or (even better) contribute bookmarks as well as join in discussion join the group. Ma.gnolia makes joining insanely simple by using OpenID for account creation and login (should you be part of the modern web world - such as having an AOL AIM account).

Does IBM Get Folksonomy?

While I do not aim to be snarky, I often come off that way as I tend to critique and provide criticism to hopefully get the bumps in the road of life (mostly digital life) smoothed out. That said...

Please Understand What You Are Saying

I read an article this morning about IBM bringing clients to Second Life, which is rather interesting. There are two statements made by Lee Dierdorff and Jean-Paul Jacob, one is valuble and the other sinks their credibility as I am not sure they grasp what they actually talking about.

The good comment is the "5D" approach, which combines the 2D world of the web and the 3D world of Second Life to get improved search and relevance. This is worth some thinking about, not a whole lot as the solution as it is mentioned can have severe problems scaling. The solution of a virtual world is lacking where it does not augment our understanding much beyond 2D as it leaves out 4 of the 6 senses (it has visual and audio), and provides more noise into a pure conversation than a video chat with out the sensory benefits of video chat. The added value of augmented intelligence via text interaction is of interest.

I am not really sure that Lee Dierdorff actually gets what he is saying as he shows a complete lack of even partial understanding of what folksonomy is. Jacob states, "The Internet knows almost everything, but tells us almost nothing. When you want to find a Redbook, for instance, it can be very hard to do that search. But the only real way to search in 5D is to put a question to others who can ask others and the answer may or may not come back to you. It's part of social search. Getting information from colleagues (online) -- that's folksonomy." Um, no that is not folksonomy and not remotely close. It is something that stands apart and is socially augmented search that can viably use the diverse structures of a folksonomy to find relevant information, but asking people in a digital world for advise is not folksonomy. It has value and it is how many of us have used tools like Twitter and other social software that helps us keep those near in thought close (see Local InfoCloud). There could be a need for a term/word for that Jacob is talking about, but social search seems to be quite relevant as a term.

Related, I do have a really large stack of criticism for the IMB DogEar product that would improve it greatly. It needs a lot of improvement as a social bookmarking and folksonomy tool, but also from the social software interaction side there are things that really must get fixed for privacy interests in the enterprise before it really could be a viable solution. There are much better alternatives for social bookmarking inside an enterprise other than DogEar, which benefits from being part of the IBM social software stack Lotus Connections as the whole stack is decent together, but none of the parts are great, or even better than good by them self. DogEar really needs to get to a much more solid product quickly as their is a lot of interest now for this type of product, but it is only a viable solution if one is only looking at IBM products for solutions.

Understanding Taxonomy and Folksonmy Together

I deeply appreciate Joshua Porter's link to from his Taxonomies and Tags blog post. This is a discussion I have quite regularly as to the relation and it is in my presentations and workshops and much of my tagging (and social web) training, consulting, and advising focusses on getting smart on understanding the value and downfalls of folksonomy tagging (as well as traditional tagging - remember tagging has been around in commercial products since at least the 1980s). The following is my response in the comments to Josh' post...

Response to Taxonomy and Tags

Josh, thanks for the link. If the world of language were only this simple that this worked consistently. The folksonomy is a killer resource, but it lacks structure, which it crucial to disambiguating terms. There are algorithmic ways of getting close to this end, but they are insanely processor intensive (think days or weeks to churn out this structure). Working from a simple flat taxonomy or faceted system structure can be enabled for a folksonomy to adhere to.

This approach can help augment tags to objects, but it is not great at finding objects by tags as Apple would surface thousands of results and they would need to be narrowed greatly to find what one is seeking.

There was an insanely brilliant tool, RawSugar [(now gone thanks to venture capitalists pulling the plug on a one of a kind product that would be killer in the enterprise market)], that married taxonomy and folksonomy to help derive disambiguation (take appleseed as a tag, to you mean Johnny Appleseed, appleseed as it relates to gardening/farming, cooking, or the anime movie. The folksonomy can help decipher this through co-occurrence of terms, but a smart interface and system is needed to do this. Fortunately the type of system that is needed to do this is something we have, it is a taxonomy. Using a taxonomy will save processor time, and human time through creating an efficient structure.

Recently I have been approached by a small number of companies who implemented social bookmarking tools to develop a folksonomy and found the folksonomy was [initially] far more helpful than they had ever imagined and out paced their taxonomy-based tools by leaps and bounds (mostly because they did not have time or resources to implement an exhaustive taxonomy (I have yet to find an organization that has an exhaustive and emergent taxonomy)). The organizations either let their taxonomist go or did not replace them when they left as they seemed to think they did not need them with the folksonomy running. All was well and good for a while, but as the folksonomy grew the ability to find specific items decreased (it still worked fantastically for people refinding information they had personally tagged). These companies asked, "what tools they would need to start clearing this up?" The answer a person who understands information structure for ease of finding, which is often a taxonomist, and a tool that can aid in information structure, which is often a taxonomy tool.

The folksonomy does many things that are difficult and very costly to do in taxonomies. But taxonomies do things that folksonomies are rather poor at doing. Both need each other.

Complexity Increases as Folksonomies Grow

I am continually finding organizations are thinking the social bookmarking tools and folksonomy are going to be simple and a cure all, but it is much more complicated than that. The social bookmarking tools will really sing for a while, but then things need help and most of the tools out there are not to the point of providing that assistance yet. There are whole toolsets missing for monitoring and analyzing the collective folksonomy. There is also a need for a really good disambiguation tool and approach (particularly now that RawSugar is gone as a viable approach).

The Social Enterprise

I am just back from Enterprise 2.0 Conference held in Boston, where I presented Bottom-up All The Way Down: How Tags Help Businesses Organize (thanks to Stowe Boyd for the tantalizing session title), which was liveblog captured by Sandy Kemsley as "Enterprise 2.0: Thomas Vander Wal". I did not catch all of the conference due to some Boston business meetings and connecting with friends and meeting digi-friends whose work I really enjoy face-to-face. The sessions I made it to were good and enlightening and as always the hallway conversations were worth their weight in gold.

Ms. Perceptions and Fear Inside the Corporate Walls

Having not been at true business focussed conference in years (until the past few weeks) I was amazed with how much has changed and how much has stayed the same. I was impressed with the interest and adoption around the social enterprise tools (blogs, wikis, social bookmarking/folksonomy, etc.). But, the misperceptions (Miss Perceptions) are still around and have grown-up (Ms. Perception) and are now being documented by Forrester and others as being fact, but the questions are seemingly not being asked properly. Around the current social web tools (blogs, wikis, social bookmarking, favoriting, shared rating, open (and partially open collaboration) I have been finding little digital divide across the ages. Initially there is a gap when tools get introduced in the corporate environment. But this age gap very quickly disappears if the incredible value of the tools is made clear for peoples worklife, information workflow, and collaboration, as well as simple instructions (30 second to 3 minute videos) and simply written clear guidelines that outline acceptable use of these tools.

I have been working with technology and its adoption in corporations since the late 80s. The misperception that older people do not get technology, are foreign to the tools, and they will not ever get the technical tools has not changed. It is true that nearly all newer technologies come into the corporation by those just out of school and have relied on these tools in university to work intelligently to get their degree. But, those whom are older do see the value in the tools once they have exposure and see the value to their worklife (getting their job done), particularly if the tools are relatively simple to use and can be adopted with simple instruction (if it needs a 10 to 200 page manual and more than 15 minutes of training to start using the product effectively adoption will be low). Toby Redshaw of Motorola stated on a panel that he found in Motorola (4600 blogs and wikis and 2600 people using social bookmarking) "people of all ages adopt these tools if they understand the value connected to their work". Personally, I have seen this has always been the case in the last 20 years as this is how we got e-mail, messaging, Blackberries, web pages, word processing, digital collaboration tools (the last few rounds and the current ones), etc. in the doors of small to large organizations. I have worked in and with technically forward organizations and ones that are traditionally thought of as slow adopters and found adoption is based on value to work and ease of use and rarely based on age.

This lack of understanding around value added and (as Toby Redshaw reinforced) "competitive advantage" derived from the social tools available today for use in the enterprise is driven by fear. It is a fear of control that is lost from the top-down. But, the advantage to the company from having this information shared and easy found and used for collaboration to improve knowledge, understanding, and efficiency can not be dismissed and needs to be embraced. The competitive advantage is what is gained today, but next month or next quarter it could mean just staying even.

Getting Beyond Fear

But, what really is important is the communication and social enterprise tools are okay and add value, but the fear is overplayed, as a percentage rarely occurs, and handling the scary stuff it relatively easy to handle.

Tagging and Social Bookmarking in Enterprise

In the halls I had many conversations around tagging ranging from old school tagging being painful because the experts needed to tag things (meaning they were not doing the job as expert they were hired to do and their terms were not widely understood) all the way to the social bookmarking tools are not scaling and able to keep up with the complexity, nor need to disambiguate the terms used. But, I was really impressed with the number of organizations that have deployed some social bookmarking effort (officially or under somebody's desk) and found value (often great value).

Toby Redshaw: I though folksonomy was going to be some Bob Dillon touchy-feely hippy taxonomy thing, but it has off the chart value far and above any thing we had expected.

My presentation had 80 to 90 percent of the people there using social bookmarking tools in some manner in their organization or worklife. The non-verbal feed back as I was presenting showed interest in how to make better sense of what was being tagged, how to use it better in their business, how to integrate with their taxonomy, and how to work with the information as the tools scale. The answers to these are longer than the hour I have, they are more complex because it all depends on the tools, how they are set-up and designed, how they are used, and the structures of information inside and outside their organization.

New Profession Unfolding In Beauty and Geekery

A week or more ago I ran across the incredible video of Blaise Aguera y Arcas presentation of Photosynth and Seadragon at TEDTalks 2007. The video is stunning work of Seadragon and Photosynth bringing a collection of images to life from one or more resources.

While the video and ideas behind the tools are incredible displays of where we are today with technology and where we are heading, this caused some ideas I have been trying to get to gel to finally come together. In this video Blaise states (my own transcription):

So what the point here really is, is we can do things with the social environment taking data from everybody, from the entire collective memory of what the earth looks like, and link all of that together and make something emergent that is greater than the sum of the parts. You have a model that emerges of the entire earth, think of it as the long tail to Stephen Lawlers Virtual Earth work. This is something that grows in complexity as people use it and whose benefits become greater to the users as they use it. Their own photos are getting tagged with metadata that somebody else entered. If somebody bothered to tag all of these saints and say who they all are, then my photo of the Notre Dame Cathedral suddenly gets enriched with all of that data. I can use it as an entry point to dive into that space in that metaverse, using everybody else's photos, and do a cross-modal and cross-user social experience that way. Of course a by product of all of that is an immensely rich virtual models of every interesting part of the earth, collected not just from overhead flight and satellite images, but from the collective memory.

Torrent of Human Contributed Digital Content

What this brought together was the incredible amount of human contributed digital content we are sitting on top of at this moment in time. This is not a new concept, but what is different is the skills, tools, and understanding to make use and sense of all this content are having to change incredibly. The Photosynth team is making use of Flickr content that has been annotated by humans (tags, titles, and descriptions), as well as by devices (date, time, location, etc.). This meta information provides hooks put pull disparate information back from its sole beauty and make an even greater beauty and deeper understanding. The collective is better than the pieces, but pulling to collective together in a manner that is coherent, adds value, and brings deeper appreciation is where get hard.

Much of information understanding and sense making to date has relied on human understanding and we have used tools to augment our understanding. But, we now need to rely on deeper analytics in quantitative methods and advanced algorithms to make sense and beauty out of the bits and bytes. The models of understanding are changing to requiring visualizations methods (much like those of Stamen Design) to begin to grasp and see what is happening in our torrent of information at our finger tips and well as make sense of the social interactions of our digitally networked and digitally augmented lives.

Amalgamation of Designer and Quant Geek

What gelled in my mind watching the Blaise demonstration is there is a skill set missing in the next generation comprised of amalgamated design, information use, analytical foundation, and strong quantitative skills. I have clients in start-up businesses and in enterprise that are confronting these floods of information they need to make sense of from folksonomies and customer generated content (including annotations and regular feedback). The skills needed for building taxonomies are not translating well when the volume of information the information managers are dealing with is orders of magnitude higher than what they dealt with previously. The designer, information architect, and taxonomist who have traditionally have dealt with building the systems of information order, access, and use are missing the quantitative skills to analyze and make sense out of a torrent of loosely structured information and digital objects. Those with the quantitative and strong analytical skills have lacked the design and art skills to bring the understanding into frame for regular people grasp and understand.

This class of designer and quant geek is much like the renaissance men, but today the field of those forging new ground is open to men and women. The need to understand not only broad but deep sets of data and information so to contextualize it into understanding is the realm of few, unfortunately as there is a need for many.

I know of limited pockets of people with the skills to do the hard work of querying the vast array of information, objects, and raw data then make something of value of it. But, there needs to be more of these people getting trained as designers with solid quantitative and analytical skills (or the converse). Design shops are missing the quant geeks and engineering shops are missing the visualization geeks that bring this digital world rich in opportunity into something that makes sense and beauty.

If you know people like this that are bored, please let me know as I am finding opportunities flowing.

Full-Day Social Web & Folksonomy Workshop at d.construct

Tickets for the d.construct Workshops go on sale June 14, 2007. Buying a ticket for a full-day workshop provides free access to the full d.construct conference on September 7th. I am presenting the following workshop on September 6th...

Building the Social Web with Tagging / Folksonomies

Folksonomy Provides 70 Percent More Terms Than Taxonomy

While at the WWW Conference in Banff for the Tagging and Metadata for Social Information Organization Workshop and was chatting with Jennifer Trant about folksonomies validating and identifying gaps in taxonomy. She pointed out that at least 70% of the tags terms people submitted in Steve Museum were not in the taxonomy after cleaning-up the contributions for misspellings and errant terms. The formal paper indicates (linked to in her blog post on the research more steve ... tagger prototype preliminary analysis) the percentage may even be higher, but 70% is a comfortable and conservative number.

Is 70% New Terms from Folksonomy Tagging Normal?

In my discussion with enterprise organizations and other clients that are looking to evaluate their existing tagging services, have been finding 30 percent to nearly 70 percent of the terms used in tagging are not in their taxonomy. One chat with a firm who had just completed updating their taxonomy (second round) for their intranet found the social bookmarking tool on their intranet turned up nearly 45 percent new or unaccounted for terms. This firm knew they were not capturing all possibilities with their taxonomy update, but did not realize their was that large of a gap. In building their taxonomy they had harvested the search terms and had used tools that analyzed all the content on their intranet and offered the terms up. What they found in the folksonomy were common synonyms that were not used in search nor were in their content. They found vernacular, terms that were not official for their organization (sometimes competitors trademarked brand names), emergent terms, and some misunderstandings of what documents were.

In other informal talks these stories are not uncommon. It is not that the taxonomies are poorly done, but vast resources are needed to capture all the variants in traditional ways. A line needs to be drawn somewhere.

Comfort in Not Finding Information

The difference in the taxonomy or other formal categorization structure and what people actually call things (as expressed in bookmarking the item to make it easy to refind the item) is normally above 30 percent. But, what organization is comfortable with that level of inefficiency at the low end? What about 70 percent of an organizations information, documents, and media not being easily found by how people think of it?

I have yet to find any organization, be it enterprise or non-profit that is comfortable with that type of inefficiency on their intranet or internet. The good part is the cost is relatively low for capturing what people actually call things by using a social bookmarking tool or other folksonomy related tool. The analysis and making use of what is found in a folksonomy is the same cost of as building a taxonomy, but a large part of the resource intensive work is done in the folksonomy through data capture. The skills needed to build understanding from a folksonomy will lean a little more on the analytical and quantitative skills side than the traditional taxonomy development. This is due to the volume of information supplied can be orders of magnitude higher than the volume of research using traditional methods.

Folksonomy Book In Progress

Let me start with an announcement. I have not had any answer for continual question after I present on tagging and folksonomy (I also get the question after the Come to Me Web and Personal InfoCloud presentations), which is "where is your book?" Well I finally have an answer, I have signed with O'Reilly to write a book, initially titled Understanding Folksonomy (this may change) and it may also be a wee bit different from your normal O'Reilly book (we will see how it goes).

I am insanely excited to be writing for O'Reilly as I have a large collection of their books from over the years - from the programming PHP, Perl, Python, and Ruby to developer guides on JavaScript, XHTML, XML, & CSS to the Polar Bear book on information architecture, Information Dashboard Design, and Designing Interfaces: Patterns for Effective Interaction Design along with so many more.

Narrowing the Subject

One of the things that took a little more time than I realized it would take is narrowing the book down. I have been keeping a running outline of tagging and folksonomy related subjects that have been in my presentations and workshop as well as questions and answers from these sessions. The outline includes the deep knowledge that has some from client work on the subject (every single client has a different twist and set of constraints. Many of the questions have answers and for some the answers are still being worked out, but the parameters and guidelines are known to get to viable solutions.

Well, when I submitted the outline I was faced with the knowledge that I had submitted a framework for a 800 to 1,000 page book. Huh? I started doing the math based on page size, word counts, bullets in the outline, and projects words per bullet and those with knowledge were right. So, I have narrowed it down to an outline that should be about 300 pages (maybe 250 and maybe a few more than 300).

What Is In This Smaller Book?

I am using my tagging and folksonomy presentations as my base, as those have been iterated and well tested (now presented some version of it well over 50 times). While I have over 300 design patterns captured from tagging service sites (including related elements) only a few will likely be used and walked through. I am including the understanding from a cognitive perspective and the lessening of technology pain that tagging services can provide to people who use them. There will also be a focus on business uses for intranet and web.

When Will This Be Done

Given that I have a rather busy Fall with client work, workshops, and presentations I set a goal to finish the writing by the end of Summer. It sounds nuts and it really feels like grad school all over again, but that is my reality. I have most of the words in my head and have been speaking them. Now I need to write them (in a less dense form than I blog).

Your Questions and Feedback

If you have questions and things you would like covered please e-mail me (contact in the header nav above). I will likely be setting up a blog to share and post questions (this will happen in a couple weeks). I am also looking for sites, organizations, and people that would like to be included in the case studies and interviews (not all will end up in the book, but those that are done will end up tied to the book in some manner). Please submit suggestions for this section if you have any.

Hoping About

Things have been a wee bit busy the past few weeks. I am off to Banff tomorrow (Monday) and boomerang right back after the WWW Conference Tagging Workshop as I have two days of private workshops in DC just following days.

I am off to the 2007 Identity Workshop for a few days, in part to talk to the some TagCommons people face to face, as well as have some client work to follow-up on. I am then off to south of the boarder, for some work before heading back to the office.

Tagging Workshop at d.construct

I should let you know early that I will be doing a full-day tagging workshop prior to d.construct 2007 in Brighton, England in September. I will post more about this as it is announced. The space is rather limited so keep an eye out.

Workshops Near You & Packages

Those interested in the workshop and I will add you to a list for announcing them. If you would like one in your organization or city I can help get this moving. Toward the end of May I will be providing more information about these, the getting smart packages (workshop or presentation combined with days of advising over a set of months). As I have mentioned these to people the interest is quite strong, so if you want more info on these before they are public on the InfoCloud Solutions site drop me an e-mail.

E-mail

If you have an e-mail that may have an answer that would be longer than 2 minutes to write I likely still have it lined up, but if you want to jump that line, send the e-mail again (or a series of short e-mails with single questions). I am deeply sorry for my delay in responding. I have no excuse, but the need for more time.

Tagging That Works Presentation Links

Today#&039;s presentation at the O'Reilly Web 2.0 Expo seemed to go very well. My session was Tagging That Works had really good feedback, which I thought was good as I really did not know the audience coming in to the presentation.

The presentation can be downloaded as PDF from Tagging That Works or can be viewed on Tagging That Works at SlideShare or view below...

From the HQ Office

It has been a good stretch of travel, mostly for work/professional life, but also took a trip to Florida for family holiday. At the moment I am back in the office working on proposals for upcoming projects of various lengths for clients and working through the process of writing, which involves dead trees at the final stages.

Web 2.0 Expo and SF Bay Area

I am soon off again to the San Francisco Bay Area to speak at the O'Reilly Web 2.0 Expo and have business meetings around the Bay Area (please ping via e-mail if you would like to meet-up during this time).

WWW2007 Workshop on Tagging and Metadata

Early next month I am off to Banff to keynote the WWW2007 Workshop on Tagging and Metadata for Social Information Organization. I am not sure how long this trip will be as I will have some pressing work around this time.

Social Software Summit

Lastly, I should note there will be a Social Software Summit that will run at the same time as the ASIS&T IA Summit next Spring (Spring 2008) in Miami. The Social Software Summit is still in its early stages of planning (the idea, dates, location, and interest have been launched). The dates are April 10-11 2008. I have a role in the planning and preparation for this event, along with some other incredible people.

On SXSW Tag You're It Panel

I am a panelist on the Tag You're It Panel at South by SouthWest in Austin, Texas. Others on the panel are the ever fantastic: Heath Row (moderator), George Oates, and Ben Brown. The panel information:

Tag You're It on Saturday 10 March 2007 at 2:00 pm to 3:00 pm. The panel will be looking at what people are actually doing inside social tagging systems and where things have gone in the past couple years with tagging. We will get beyond the notion that tagging is cool by providing examples of how people are really using the tools in innovative and useful ways.

Stop by and say hello.