PHP output showing little black diamonds with a qu

Question

I'm writing a php program that pulls from a database source. Some of the varchars have quotes that are displaying as black diamonds with a question mark in them (�, REPLACEMENT CHARACTER, I assume from Microsoft Word text).

How can I use php to strip these characters out?

Answer 1

Try This Please

mb_substr($description, 0, 490, "UTF-8");

Answer 2

what I ended up doing in the end after I fixed my tables was to back it up and change back the settings to utf-8 then I altered my dump file so that DEFAULT CHARACTER SET utf8 COLLATE utf8_general_ci are my character set entries

now I don't have characterset issues anymore because the database and browser are utf8.

I figured out what caused it. It was the web page+browser effects on the DB. On the terminals that are linux (ubuntu+firefox) it was encoding the database in latin1 which is what the tabes are set. But on the windows 10+edge terminals, the entries were force coded into utf8. Also I noticed the windows 10 has issues staying with latin1 so I decided to bend with the wind and convert all to utf8.

I figured it was a windows 10 issue because we started using win 10 terminals. so yet again microsoft bugs causes issues. I still don't know why the encoding changes on the forms because the browser in windows 10 shows the latin1 characterset but when it goes in its utf8 encoded and I get the data anomaly. but in linux+firefox it doesn't do that.

Answer 3

You can also change the caracter set in your browser. Just for debug reasons.

Answer 4

When you extract data from anywhere you should use functions with the prefix md_FUNC_NAME.

Had the same problem it helped me out.

Or you can find the code of this symbol and use regexp to delete these symbols.

Answer 5

That can be caused by unicode or other charset mismatch. Try changing charset in your browser, in of the settings the text will look OK. Then it's question of how to convert your database contents to charset you use for displaying. (Which can actually be just adding utf-8 charset statement to your output.)

Answer 6

This is a charset issue. As such, it can have gone wrong on many different levels, but most likely, the strings in your database are utf-8 encoded, and you are presenting them as iso-8859-1. Or the other way around.

The proper way to fix this problem, is to get your character-sets straight. The simplest strategy, since you're using PHP, is to use iso-8859-1 throughout your application. To do this, you must ensure that:

• All PHP source-files are saved as iso-8859-1 (Not to be confused with cp-1252).
• Your web-server is configured to serve files with charset=iso-8859-1
• Alternatively, you can override the webservers settings from within the PHP-document, using header.
• In addition, you may insert a meta-tag in you HTML, that specifies the same thing, but this isn't strictly needed.
• You may also specify the accept-charset attribute on your <form> elements.
• Database tables are defined with encoding as latin1
• The database connection between PHP to and database is set to latin1

If you already have data in your database, you should be aware that they are probably messed up already. If you are not already in production phase, just wipe it all and start over. Otherwise you'll have to do some data cleanup.

A note on meta-tags, since everybody misunderstands what they are:

When a web-server serves a file (A HTML-document), it sends some information, that isn't presented directly in the browser. This is known as HTTP-headers. One such header, is the Content-Type header, which specifies the mimetype of the file (Eg. text/html) as well as the encoding (aka charset). While most webservers will send a Content-Type header with charset info, it's optional. If it isn't present, the browser will instead interpret any meta-tags with http-equiv="Content-Type". It's important to realise that the meta-tag is only interpreted if the webserver doesn't send the header. In practice this means that it's only used if the page is saved to disk and then opened from there.

This page has a very good explanation of these things.