Upgrade from 2.8.3 to 2.10.0-latin1 to utf8

WebsiteBaker Support (2.10.x) > General Help & Support

<< < (3/6) > >>

contactjw:
#6 examples are from pages in ckeditor in source code view.

My database is latin1_swedish_ci. All my tables are latin1_swedish_ci except for droplets, output filter, and settings which are all utf8_unicode_ci. Not sure why it's swedish? :?

Here's how my db file looks:

(Addons table latin1)

CREATE TABLE `wbaddons` (
`addon_id` int(11) NOT NULL AUTO_INCREMENT,
`type` varchar(255) NOT NULL DEFAULT '',
`directory` varchar(255) NOT NULL DEFAULT '',
`name` varchar(255) NOT NULL DEFAULT '',
`description` text NOT NULL,
`function` varchar(255) NOT NULL DEFAULT '',
`version` varchar(255) NOT NULL DEFAULT '',
`platform` varchar(255) NOT NULL DEFAULT '',
`author` varchar(255) NOT NULL DEFAULT '',
`license` varchar(255) NOT NULL DEFAULT '',
PRIMARY KEY (`addon_id`),
UNIQUE KEY `ident` (`directory`)
) ENGINE=MyISAM AUTO_INCREMENT=198 DEFAULT CHARSET=latin1;
/*!40101 SET character_set_clien t = @saved_cs_client */;

(Settings table utf8.)

CREATE TABLE `wbsettings` (
`name` varchar(255) COLLATE utf8_unicode_ci NOT NULL DEFAULT '',
`value` text COLLATE utf8_unicode_ci NOT NULL,
PRIMARY KEY (`name`)
) ENGINE=MyISAM DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci;
/*!40101 SET character_set_clien t = @saved_cs_client */;

So if I change all the default latin1 entries to utf 8 will that be enough?

contactjw:
I changed the db to utf8_unicode_ci. Then I changed all the latin1 entries to utf8 in notepad++. Now all my tables show utf8_general_ci except the ones that were already utf8_unicode_ci. They stayed the same. I also used the ckeditor to change the few remaining entries like â€™ that I missed when I first did the ones that were not reading properly. Everything seems to be working ok.

Did I do anything wrong? Is there anything else I need to do?

Gast:
what is the used ckeditor version and do you use the editor setting $ckeditor->config['entities'] = true ??

you found this setting (if avaiable) in include.php of the editor (modules/ckeditor/include.php), nearly in the middle of the file, in my version at line 183

or the same setting in wb_ckconfig.js
config.entities = false;

to explain it: this option converted (if TRUE) special chars to html-entities (for example: the apostroph ' is saved as ' )
is this option deactivated or false, the script use the charset from WB, save it in sql-language as â€™ and in the browser readable as '

pls check the table collation of this table. open this section with the ckeditor, search there for the section-id in the head of this section and search in the module-table mod_wysiwyg (if its is a wysiwyg section). what do you see in the database tool like phpmyadmin?
export this table and open it with a simple editor like notepad (not notepad++) what do you see there???

contactjw:
Version 4.6.2.

if (!$bWbConfigSetting ) { $ckeditor->config['entities'] = false; }

config.entities = false;

(This is from a news module post)

When the road you're trudging seems all uphill, (ckeditor source view)

When the road you’re trudging seems all uphill, (phpmyadmin content view)

When the road you’re trudging seems all uphill, (notepad)

(This is from a normal page.)

<a href="https://en.wikipedia.org/wiki/Fran%C3%A7ois_Cluzet" target="_blank">François Cluzet</a> (ckeditor source view)

<a href="https://en.wikipedia.org/wiki/Fran%C3%A7ois_Cluzet" target="_blank">François Cluzet</a> (phpmyadmin content view)

<a href=\"https://en.wikipedia.org/wiki/Fran%C3%A7ois_Cluzet\" target=\"_blank\">François Cluzet</a> \n (notepad)

Also, while looking at phpmyadmin I noticed that my localhost db is set to latin1 swedish with all the tables the same as my wb db was before I changed it. Should I change my localhost db like I did my wb db?

Gast:
thanks for the infos, but i see, that the simple notepad doesnt show the real SQL :x sorry for the trouble

here the real UTF8-Chars from the table wb_addons in real SQL without Converter, only visible in a editor, but not in your browser

and the same code in notepad, notepad++ or other IDE's with included converter

but back to your post

the editors like ckeditor, tiny_mc etc has a own converter included, if activated, the editor convert special chars via html_entities(), a php-function. i dont now, how much, but html_entities has a small range, not possible, to convert all chars from all over the world. thats why, utf8 is the better solution, there are no limits. because of that, the php-group make a recommendation to use utf8 as default charset on every server since PHP 5.5. the most providers follow this instruction. in older php-versions it was the choice of the provider. german providers prefer the latin-charsets in the past and today its nearly not possible, to change the default charset without trouble. if they change the server charset for all the customers at the same time, you have a big chaos on the service line on one day or one week, so its part of the customers, to choose a new charset (like your case), so you have maybe only 1 user with trouble every day and not all in the same time :wink:

to your code-examples
thats okay in ckeditor source view and also the version with entities is okay in ckeditor source view

--- Quote ---When the road you're trudging seems all uphill, (ckeditor source view)
--- End quote ---

the editor and his own converter convert the entities from the database and show the real char (like your apostroph). it doesnt matter, what the script found in the database. if its a readable char, its possible to convert it
P.S.: if the editor founds a not-readable char and its not possibe to convert it, the editor show nothing in the editor field and show also no toolbar. its a javascript error and its not possible for javascript to show not-existing-chars

But.... whats happend, if you save this content??

--- Quote ---When the road you’re trudging seems all uphill, (phpmyadmin content view)
--- End quote ---

after saving this content from your example (with the correct charset and correct table collation) the phpmyadmin must show the real chars like

--- Quote ---When the road you're trudging seems all uphill,
--- End quote ---

its okay for older CKeditor versions with activated entities-converter, but not for the newer version. if i remenber correct, we change the default settings for the ckeditor in WB 2.8.3 SP6, but its possible for every User, to use the old method, if he set config.entities to TRUE
Background for this was the view in tools like phpmyadmin. its difficult to read the content, when i use Entities like this

--- Quote ---When the road you’re trudging seems all uphil
--- End quote ---
Without entities, i've the same view in phpmyadmin like my output in Front- or backend, the used charset convert the SQL-language to readable chars without entities

notepad++ has a good converter, but not possible to convert different methods at the same time, because of that, we use php-converters, in my case with a big array of chars to replace. Maybe you can send me your zipped sql-backup and i'll try to convert it.

for my locale development, i use xampp for windows with the default settings from xampp. if i look into phpmyadmin at the databases overview, i found in the last line latin1_swedish_ci, dont know, where it comes from. the same on the online-servers, dont know why, maybe a default setting somewhere.

in one project, i've a lot of trouble like yours, a long period of search&&try, but then i found the database charset settings in this overview and it was set to latin1 also, so i change it to utf8_general_ci and all the problems are gone after converting all the chars in the backup
to change it, go to phpadmin, startpage with the overview (show all databases as list in the big, right content-field, (like my last picture). Click on the link to your prefered database and after reload in the top-menu on the OPERATION-Link, there you can change the collation for this database

Navigation

[0] Message Index

[#] Next page

[*] Previous page

Go to full version