{var%20f='http://v.t.sina.com.cn/share/share.php?appkey=1515056452',u=z||d.location,p=['&url=',e(u),'&title=',e(t||d.title),'&source=',e(r),'&sourceUrl=',e(l),'&content=',c||'gb2312','&pic=',e(p||'')].join('');function%20a(){if(!window.open([f,p].join(''),'mb',['toolbar=0,status=0,resizable=1,width=440,height=430,left=',(s.width-440)/2,',top=',(s.height-430)/2].join('')))u.href=[f,p].join('');};if(/Firefox/.test(navigator.userAgent))setTimeout(a,0);else%20a();})(screen,document,encodeURIComponent,'','','https://www.manongdao.com/data/attach/logo/logo.png', '推荐 看我几分像从前 的问题《_Actual_ Twitter format for hashtags? Not your reg》','https://www.manongdao.com/q-498565.html','页面编码gb2312|utf-8默认gb2312'));){kind=link}
Update: Use Twitter's Entities if you can- they figured it out for you as well as other items. My case is that I just have the tweet without entities and all the extra metadata
I've spent what I consider an unreasonable amount of time trying to find the actual format for hashtags.
As far as my searching can tell- Twitter has not published one.
I know that many people have come up with regex's to parse them, however, your lib's regex is not my lib's regex and maybe I don't like yours anyway.
So I'm asking- is there any actual official spec? I don't want a regex answer, I want a BNF or something similar. Or minimally- a complete list of delimiters.
Additional difficulty points- grabbing them from random unicode messages (non-English) text is important too.
Note: I'm quite aware of entities and they aren't applicable to my case (months of twitter messages stored in a db).
From the starting point of twitter's support the basic rules seems to be that hashtags must be preceded by a space and stop on any whitespace or punctuation.
Quote from Twitter's support:
Check your hashtags for the following:
Therefore, the initial token is
#preceded by a space, and the terminator is any whitespace or punctuation. The "etc" in their list of punctuation (" , . ; ' ? ! etc.") is annoying, but I'll keep digging and see if I can find something authoritative on what else counts as punctuation.After digging around a while, I found some interesting blog articles by Terence Eden (Hashtags and Implicit Knowledge, Hashtag Standards) that provide evidence that Twitter doesn't even have a standard, given that the software it develops on different platforms seems to have different rules of what constitutes a hashtag.
It also provided a link to the Twitter Conformance Library, which has twitter / twitter-text-conformance / autolink.yml. The
hashtagsection in autolink.yml has many cases matching the above rules, but also some that violate them are are still supposed to be autolinked. Some examples:Those are just a few examples that don't match the basic rules given in the first support article, and unfortunately the
ymlis full of other examples as well.The Twitter entity parsing libraries are available here: https://github.com/twitter/twitter-text
I found this : "Need help parsing tweet text?", on dev.twitter.com
(there's ruby, java and javascript librairies)
They are quite enormous, as twitter must take into account every possible case.
There is in fact an official specification for hashtags. Twitter accepts only a subset of Unicode expressions for the hashtag syntax. Here is the regular expression to recognize all valid Hashtags used on Twitter (pulled from their own sourcecode.)
To see how it's generated see the source code of twitter-text.
this is what I use, the closest i get:
link of the hashtag Regex to test
Based on how the official Twitter client for Mac highlights hashtags, I suspect the rule is any sequence of contiguous letters, numbers, or underlines following a hash. In other words, it's as simple as the regex
/#\w+/(assuming a unicode-aware regex engine).