Hi everybody, here's my summer trouble, if somebody can help me ... i
can send some free beer ;-)

I have a TYPO3 installation setup in UTF-8, like that:

$TYPO3_CONF_VARS['SYS']['t3lib_cs_convMethod'] = 'iconv';
$TYPO3_CONF_VARS['SYS']['t3lib_cs_utils'] = 'iconv';
$TYPO3_CONF_VARS['BE']['forceCharset'] = 'utf-8';

Is the TS template

config.renderCharset = utf-8

But i forgot to set in localconf.php the line:

$TYPO3_CONF_VARS['SYS']['setDBinit'] = 'SET NAMES utf8;';

If I add this line, the frontend (templavoila) loose the template
configuration, and in the backend all the accentuated characters as
trouble.

So my MySQL client is always in latin1 (I check the configuration of
mysql with SHOW VARIABLES, and it's the case)

Everything work nice in the TYPO3 backend and frontend, but if I
insert data in the TYPO3 database with phpmyadmin or an other mysql
client correctly configured in UTF8, the output in TYPO3 is not good

When i browse my database in PHPMyAdmin, i can't see the accentuated
characters correctly except if I edit this record with phpmyadmin.
But in this case TYPO as some trouble with the accentuated characters.

If I understand well the problem, TYPO3 send UTF8 data, but the MySQL
client translate it lik if the data was in latin1 to UTF8 (the
database use utf8_general_ci as collation). finaly my data are not in
UTF8 nor in latin1 ...

So my 2 cents question, is how can I translate/convert this DB to
have a standard UTF-8 configuration.

I have a lots of data inside the database so a manual edition is not
possible

Thanks a lots for your help

Dominique Feyer



_______________________________________________
TYPO3-english mailing list
TYPO3-english (AT) lists (DOT) netfielders.de
http://lists.netfielders.de/cgi-bin/.../typo3-english

Hi Dominique,

Dominique Feyer wrote:
> Hi everybody, here's my summer trouble, if somebody can help me ... i
> can send some free beer ;-)
>
> I have a TYPO3 installation setup in UTF-8, like that:
>
> $TYPO3_CONF_VARS['SYS']['t3lib_cs_convMethod'] = 'iconv';
> $TYPO3_CONF_VARS['SYS']['t3lib_cs_utils'] = 'iconv';
> $TYPO3_CONF_VARS['BE']['forceCharset'] = 'utf-8';
>
> Is the TS template
>
> config.renderCharset = utf-8
>
> But i forgot to set in localconf.php the line:
>
> $TYPO3_CONF_VARS['SYS']['setDBinit'] = 'SET NAMES utf8;';
>
> [...]
>
> So my 2 cents question, is how can I translate/convert this DB to have a
> standard UTF-8 configuration.
>
> I have a lots of data inside the database so a manual edition is not
> possible

Give it a try:
http://xavier.perseguers.ch/en/tutor...ion/utf-8.html

--
Xavier Perseguers
http://xavier.perseguers.ch/en
_______________________________________________
TYPO3-english mailing list
TYPO3-english (AT) lists (DOT) netfielders.de
http://lists.netfielders.de/cgi-bin/.../typo3-english

Hi!

Dominique Feyer wrote:
> $TYPO3_CONF_VARS['SYS']['setDBinit'] = 'SET NAMES utf8;';
>
> If I add this line, the frontend (templavoila) loose the template
> configuration, and in the backend all the accentuated characters as
> trouble.

Yes. MySQL starts returning data differently. TemplaVoila cannot decode it anymore. You need to remap templates.

> Everything work nice in the TYPO3 backend and frontend, but if I insert
> data in the TYPO3 database with phpmyadmin or an other mysql client
> correctly configured in UTF8, the output in TYPO3 is not good

What do you mean by "correctly configured"?

> When i browse my database in PHPMyAdmin, i can't see the accentuated
> characters correctly except if I edit this record with phpmyadmin. But
> in this case TYPO as some trouble with the accentuated characters.

I guess phpMyAdmin does not make "set names utf8" automatically for you. This is why you see data in the wrong way.

--
Dmitry Dulepov
TYPO3 Core team
My TYPO3 book: http://www.packtpub.com/typo3-extens...velopment/book
In the blog: http://typo3bloke.net/post-details/t...ng_in_eclipse/
_______________________________________________
TYPO3-english mailing list
TYPO3-english (AT) lists (DOT) netfielders.de
http://lists.netfielders.de/cgi-bin/.../typo3-english

Dominique Feyer wrote:
> Hi everybody, here's my summer trouble, if somebody can help me ... i
> can send some free beer ;-)
>
> I have a TYPO3 installation setup in UTF-8, like that:
>
> $TYPO3_CONF_VARS['SYS']['t3lib_cs_convMethod'] = 'iconv';
> $TYPO3_CONF_VARS['SYS']['t3lib_cs_utils'] = 'iconv';
> $TYPO3_CONF_VARS['BE']['forceCharset'] = 'utf-8';
>
> Is the TS template
>
> config.renderCharset = utf-8
>
> But i forgot to set in localconf.php the line:
>
> $TYPO3_CONF_VARS['SYS']['setDBinit'] = 'SET NAMES utf8;';

So, you are running a live system all the time and now you changed the
setDBinit?

> If I understand well the problem, TYPO3 send UTF8 data, but the MySQL
> client translate it lik if the data was in latin1 to UTF8 (the database
> use utf8_general_ci as collation). finaly my data are not in UTF8 nor in
> latin1 ...

No, the problem now occurs because you added SET NAMES utf8. Your former
configuration without SET NAMES worked fine in itself. The configuration
was wrong when you liked it to be completly in utf8, but in itself it
worked. - We had the same thing on one server.

> So my 2 cents question, is how can I translate/convert this DB to have a
> standard UTF-8 configuration.

We had a similar problem with all our TYPO3 Installs on a certain
server. Doing a convert on the fly isnt that funny - I would suggest
leaving SET NAMES out and live with this misconfigured system. As I
said: it works.

If you want to convert you have to alter the DB character set first.
Then change collations of the tables (needed for searching). Then
convert all content in all relevant tables. Truncate caching tables. The
last part is go through your TypoScript and check if those are also
converted.

> I have a lots of data inside the database so a manual edition is not
> possible

I could send you a script I made for this purpose, but it only works if
dont already have mixed up the encoding. Meaning if you have content in
your DB that was inserted without SET NAMES utf8 in you localconf and
content that was insertet with SET NAMES utf8 ... you have a problem.

> Thanks a lots for your help
>
> Dominique Feyer

Greetings
Lars
_______________________________________________
TYPO3-english mailing list
TYPO3-english (AT) lists (DOT) netfielders.de
http://lists.netfielders.de/cgi-bin/.../typo3-english

Le 18 août 08 à 12:05, Dmitry Dulepov [typo3] a écrit :

> Hi!
>
> Dominique Feyer wrote:
>> $TYPO3_CONF_VARS['SYS']['setDBinit'] = 'SET NAMES utf8;';
>>
>> If I add this line, the frontend (templavoila) loose the template
>> configuration, and in the backend all the accentuated characters as
>> trouble.
>
> Yes. MySQL starts returning data differently. TemplaVoila cannot
> decode it anymore. You need to remap templates.
>
>> Everything work nice in the TYPO3 backend and frontend, but if I
>> insert
>> data in the TYPO3 database with phpmyadmin or an other mysql client
>> correctly configured in UTF8, the output in TYPO3 is not good
>
> What do you mean by "correctly configured"?

I mean the client configured as UTF8 (like 'SET NAMES utf8;' for
phpmyadmin, or UTF8 for ODBC)

>
>> When i browse my database in PHPMyAdmin, i can't see the accentuated
>> characters correctly except if I edit this record with phpmyadmin.
>> But
>> in this case TYPO as some trouble with the accentuated characters.
>
> I guess phpMyAdmin does not make "set names utf8" automatically for
> you. This is why you see data in the wrong way.

I see the data correctly in the typo3 backend but not in PHPMyAdmin
(configured as UTF8) and external database tool connected over ODBC
as UTF8

>
> --
> Dmitry Dulepov
> TYPO3 Core team
> My TYPO3 book: http://www.packtpub.com/typo3-extension-development/
> book
> In the blog: http://typo3bloke.net/post-details/
> typo3_code_formatting_in_eclipse/
> _______________________________________________
> TYPO3-english mailing list
> TYPO3-english (AT) lists (DOT) netfielders.de
> http://lists.netfielders.de/cgi-bin/.../typo3-english

_______________________________________________
TYPO3-english mailing list
TYPO3-english (AT) lists (DOT) netfielders.de
http://lists.netfielders.de/cgi-bin/.../typo3-english

Le 18 août 08 à 13:26, Lars Haendler a écrit :

> Dominique Feyer wrote:
>> Hi everybody, here's my summer trouble, if somebody can help me ... i
>> can send some free beer ;-)
>>
>> I have a TYPO3 installation setup in UTF-8, like that:
>>
>> $TYPO3_CONF_VARS['SYS']['t3lib_cs_convMethod'] = 'iconv';
>> $TYPO3_CONF_VARS['SYS']['t3lib_cs_utils'] = 'iconv';
>> $TYPO3_CONF_VARS['BE']['forceCharset'] = 'utf-8';
>>
>> Is the TS template
>>
>> config.renderCharset = utf-8
>>
>> But i forgot to set in localconf.php the line:
>>
>> $TYPO3_CONF_VARS['SYS']['setDBinit'] = 'SET NAMES utf8;';
>
> So, you are running a live system all the time and now you changed the
> setDBinit?

Yes exactly

>
>> If I understand well the problem, TYPO3 send UTF8 data, but the MySQL
>> client translate it lik if the data was in latin1 to UTF8 (the
>> database
>> use utf8_general_ci as collation). finaly my data are not in UTF8
>> nor in
>> latin1 ...
>
> No, the problem now occurs because you added SET NAMES utf8. Your
> former
> configuration without SET NAMES worked fine in itself. The
> configuration
> was wrong when you liked it to be completly in utf8, but in itself it
> worked. - We had the same thing on one server.
>
>> So my 2 cents question, is how can I translate/convert this DB to
>> have a
>> standard UTF-8 configuration.
>
> We had a similar problem with all our TYPO3 Installs on a certain
> server. Doing a convert on the fly isnt that funny - I would suggest
> leaving SET NAMES out and live with this misconfigured system. As I
> said: it works.

Yes it work if i use only TYPO3 to access the database, but some data
are send with Filemaker over ODBC (in UTF8) and in this case TYPO3
can't handle correctly this content (accent problem, and special
caracter)

>
> If you want to convert you have to alter the DB character set first.
> Then change collations of the tables (needed for searching). Then
> convert all content in all relevant tables. Truncate caching
> tables. The
> last part is go through your TypoScript and check if those are also
> converted.
>
>> I have a lots of data inside the database so a manual edition is not
>> possible
>
> I could send you a script I made for this purpose, but it only
> works if
> dont already have mixed up the encoding. Meaning if you have
> content in
> your DB that was inserted without SET NAMES utf8 in you localconf and
> content that was insertet with SET NAMES utf8 ... you have a problem.

I have only data without the SET NAMES utf8 it try to add this
configuration only a on staging environnement

>
>> Thanks a lots for your help
>>
>> Dominique Feyer
>
> Greetings
> Lars
> _______________________________________________
> TYPO3-english mailing list
> TYPO3-english (AT) lists (DOT) netfielders.de
> http://lists.netfielders.de/cgi-bin/.../typo3-english

_______________________________________________
TYPO3-english mailing list
TYPO3-english (AT) lists (DOT) netfielders.de
http://lists.netfielders.de/cgi-bin/.../typo3-english

Dominique Feyer wrote:
>
> Le 18 août 08 à 13:26, Lars Haendler a écrit :
>
>> Dominique Feyer wrote:
>>> Hi everybody, here's my summer trouble, if somebody can help me ... i
>>> can send some free beer ;-)
>>>
>>> I have a TYPO3 installation setup in UTF-8, like that:
>>>
>>> $TYPO3_CONF_VARS['SYS']['t3lib_cs_convMethod'] = 'iconv';
>>> $TYPO3_CONF_VARS['SYS']['t3lib_cs_utils'] = 'iconv';
>>> $TYPO3_CONF_VARS['BE']['forceCharset'] = 'utf-8';
>>>
>>> Is the TS template
>>>
>>> config.renderCharset = utf-8
>>>
>>> But i forgot to set in localconf.php the line:
>>>
>>> $TYPO3_CONF_VARS['SYS']['setDBinit'] = 'SET NAMES utf8;';
>>
>> So, you are running a live system all the time and now you changed the
>> setDBinit?
>
> Yes exactly
>
>>
>>> If I understand well the problem, TYPO3 send UTF8 data, but the MySQL
>>> client translate it lik if the data was in latin1 to UTF8 (the database
>>> use utf8_general_ci as collation). finaly my data are not in UTF8 nor in
>>> latin1 ...
>>
>> No, the problem now occurs because you added SET NAMES utf8. Your former
>> configuration without SET NAMES worked fine in itself. The configuration
>> was wrong when you liked it to be completly in utf8, but in itself it
>> worked. - We had the same thing on one server.
>>
>>> So my 2 cents question, is how can I translate/convert this DB to have a
>>> standard UTF-8 configuration.
>>
>> We had a similar problem with all our TYPO3 Installs on a certain
>> server. Doing a convert on the fly isnt that funny - I would suggest
>> leaving SET NAMES out and live with this misconfigured system. As I
>> said: it works.
>
> Yes it work if i use only TYPO3 to access the database, but some data
> are send with Filemaker over ODBC (in UTF8) and in this case TYPO3 can't
> handle correctly this content (accent problem, and special caracter)
>
>>
>> If you want to convert you have to alter the DB character set first.
>> Then change collations of the tables (needed for searching). Then
>> convert all content in all relevant tables. Truncate caching tables. The
>> last part is go through your TypoScript and check if those are also
>> converted.
>>
>>> I have a lots of data inside the database so a manual edition is not
>>> possible
>>
>> I could send you a script I made for this purpose, but it only works if
>> dont already have mixed up the encoding. Meaning if you have content in
>> your DB that was inserted without SET NAMES utf8 in you localconf and
>> content that was insertet with SET NAMES utf8 ... you have a problem.
>
> I have only data without the SET NAMES utf8 it try to add this
> configuration only a on staging environnement
>
>>
>>> Thanks a lots for your help
>>>
>>> Dominique Feyer
>>
>> Greetings
>> Lars
>> _______________________________________________
>> TYPO3-english mailing list
>> TYPO3-english (AT) lists (DOT) netfielders.de
>> http://lists.netfielders.de/cgi-bin/.../typo3-english
>

I see, you have to convert. This will be fun ;-)

Greetings
Lars

_______________________________________________
TYPO3-english mailing list
TYPO3-english (AT) lists (DOT) netfielders.de
http://lists.netfielders.de/cgi-bin/.../typo3-english

Le 18 août 08 à 13:26, Lars Haendler a écrit :

> I could send you a script I made for this purpose


Can you send me your script ? I go in holiday for 2 week tomorrow
evening, and my client need a clean DB during the next week

Thanks a lots,

Dominique

_______________________________________________
TYPO3-english mailing list
TYPO3-english (AT) lists (DOT) netfielders.de
http://lists.netfielders.de/cgi-bin/.../typo3-english

Dominique Feyer wrote:
>
> Le 18 août 08 à 13:26, Lars Haendler a écrit :
>
>> I could send you a script I made for this purpose
>
>
> Can you send me your script ? I go in holiday for 2 week tomorrow
> evening, and my client need a clean DB during the next week
>
> Thanks a lots,
>
> Dominique
>
Already sent.
_______________________________________________
TYPO3-english mailing list
TYPO3-english (AT) lists (DOT) netfielders.de
http://lists.netfielders.de/cgi-bin/.../typo3-english