Get the Best WordPress Backup
& Migration Plugin Today
Get Duplicator Now

Duplicator Documentation

Documentation, Reference Materials, and Tutorials for Duplicator

How to fix unicode and character encoding issues?

Issue ACharacters/Symbols
How can I fix strange character, unicode symbols, emoji icons?

Strange characters such as ” � ” or “????” are a sign of encoding issues when you move from one server to another. If the database version changesacross servers then scripts may not work as expected. By default WordPress sets your character encoding to UTF-8 or UTF8MB4.Adjusting the database charset and collation can help clear up these issues on some servers.Here are some items you can try:

Solution 1

wpconfig.php:

Online Overview See: DB_CHARSET,DB_COLLATE

Try changing the DB_CHARSET optiondefine("DB_CHARSET", "utf8");
define("DB_COLLATE", "");
Try using no options or commenting them outdefine("DB_CHARSET","");
define("DB_COLLATE","");
//define("DB_CHARSET", "some_value");
//define("DB_COLLATE", "some_value");
Try various options UTF8MB4 to UTF8

Try utf8mb4 or vice-versa:define("DB_CHARSET", "utf8mb4");Try just utf8:define("DB_CHARSET", "utf8");
Solution 2

Installer
  1. On step 2 of installer under options
  2. Set charset: utf8mb4
  3. Set collation: utf8mb4_unicode_ci

Solution 3

General Issues

Update Meta Tag:This is usually something you can do in your header.php file.The server is not sending the charset in the Content-Type header and the HTML meta fallback is broken:
<meta http-equiv=”Content-Type” content=”text/html; charset=UTF-8?>

Note the invalid curly quotes instead of straight quotes. That means there’s no valid declaration for what encoding the page is in and hence the browser takes a guess.Some browsers guess wrong.

Fix Database: Encoding issues can get complex and if the above solutions don’t work then you may need to performa deeper dive into fixing the issue.

Consider an Upgrade:In some cases we have seen that updating to PHP 7+ along with a the latest MySQL server can solve some of these character encoding issues.After updating to these new and improved platforms be sure to re-build a new package.


Issue BUnicode
How can I fix international characters issues?

Solution 1Support for international character can vary across systems. The process of converting/retaining a character set when moving across different operatingsystems, web servers & database servers has many complexities, especially when the various versions are different across the build location and installlocation. If you run into character conversion/encoding issues below is a list of things you can try:

DATABASE ISSUES
  • Check wp-config.php
    Look for the DB_CHARSET andDB_COLLATE constants, these values can affect howWordPress handles your character encoding. If DB_COLLATE and DB_CHARSET do not exist in your wp-config.php file, try adding/editing either definitionto your wp-config.php file. Please be sure to read the following article to better understandConverting Database Character Sets.
  • Advanced Options
    In step one of the installer under advanced options there are two character values that you can try to enable which are”MySQL Charset” and “MySQL Collation” indirectly these values work withmysqli_set_charset. These values may not work on all systems.
  • Database.sql File
    Open the database.sql file in Notepad++ and selected “Encode in UTF-8″from the Encoding menu, without changing the contents at all. In some situations this can be more important than changing theDEFAULT CHARACTERSET, as it determines how the file is read by the operating system.
FILE SYSTEM

On some operating systems and version of PHP there are known issues with working with international characters. This is a limitation with the underlying PHP ZipArchive and json_encode logic used in both Lite and Pro.

  • Filter Files
    A common work-a-round at this point is to filter the international files and then manually move them later. The scanner process will help you to spotthese files.
  • Shell Zip
    In Duplicator Pro there is a new process option called shell_exec which issupported by most hosts. This processcan on most systems better handle international characters in file names. Please use this option in Pro if you’re running into file/directory names with unicode characters.
  • PHP 7+
    PHP 7+ has been improved to better handle unicode issues. If you can migrate or get your host to upgrade to the improved/faster performing version ofPHP 7 this can solve the issue for internal characters issues at many levels.

 

Please note if you do run into an issue please feel free to submit a ticket and we will do our best to help you work through any know fixes.However if the process involves upgrading your database or performing data conversion migration scripts then we recommend you work with aqualified developer to help research and diagnose your particular issue.

Duplicator only moves, it does not have the ability to repair and convert data when moving to a different server setup that does not supportthe environment it was created on. Please checkout the Duplicator Partner Page fora list of resources that specialize in data conversion.

Was this article helpful?

Related Articles