Need testers for some big changes on staging

Aug 30, 2024 11:15 pm

Ok, that's what I needed. Someone on SO said that when you convert a table, it doesn't convert the data inside. I'm gonna have to write a script that will update every forum post with it's title converted, and we should be good. Same with post bodies and any other place emojis are.

Aug 30, 2024 11:18 pm

Ok, can someone check if games from prod with emojis in the title are showing up correctly? And to confirm, any new games you created with emojis in the title aren't?

Aug 30, 2024 11:32 pm

Erm, it seems that I can see all emojis on staging now? I made two new games titled "❤️❤️❤️" and "💡💡💡 Test" respectively, and they display correctly everywhere. Same with Cyrillics (except the old one).

Edit: except in emails, it seems, those are still broken.

Last edited August 30, 2024 11:37 pm

Aug 30, 2024 11:38 pm

New data should always have been working. It's old data that's problematic. Do old games forums now show emojis in the title?

Aug 30, 2024 11:42 pm

Well, the new data with Cyrillic didn't work before for whatever reason. Works now though, so idk.

I can see the emoji in GP Develpment game for what's it worth, but I can't check if it looks fine inside since it's private. And I'm not in any games with emojis, so unfortunately I can't check that further. :(

Edit: ooh, I forgot, I am in one retired game with an emoji in title. It looks fine.

Last edited August 30, 2024 11:48 pm

Aug 30, 2024 11:55 pm

Ok, then the solution is working. So, next steps:
- Lets recap what's still wrong with staging. I know there's some data missing, please let me know what that was
- Lets list out what data has/likely has emojis so I can fix all the fields

Aug 31, 2024 12:05 am

Keleth says:

- Lets recap what's still wrong with staging. I know there's some data missing, please let me know what that was

Bookmarks for games and characters get messed up on import.
Some private games are randomly marked public (but are not functionally public).
Subscription are empty (I can't actually confirm if it's still true since I don't have any, but vagueGM reported it earlier.)
Advanced rules definitions display without line breaks, with \n instead.

[ +- ] Example

"{\n\"characterSheetIntegration\":true,\n\"diceRules\":[\n{\"rolled\": \"2d6\",\"reason\":\"SR: 5-##\",\"total\": \"5<=x<=##\",\"highlight\": \"bold success\",\"contentAppend\": \"✅\"},\n{\"rolled\": \"d6\",\"lastDie\":\"true\",\"highlight\":\"info bold invert\",\"autoSort\":\"true\",\"hideTotal\":true},\n{\"rolled\": \"D20\", \"autoSort\":true},\n{\"rolled\": \"D20\", \"natural\":20,\"highlight\": \"fail invert bold \"},\n{\"rolled\": \"D20\",\"reason\": \"TN 1\", \"natural\": \"<=1\", \"highlight\": \"success invert\", \"countTotal\": \"<=1\"},\n{\"rolled\": \"D20\",\"reason\": \"TN 2\", \"natural\": \"<=2\", \"highlight\": \"success invert\", \"countTotal\": \"<=2\"},\n{\"rolled\": \"D20\",\"reason\": \"TN 3\", \"natural\": \"<=3\", \"highlight\": \"success invert\", \"countTotal\": \"<=3\"},\n{\"rolled\": \"D20\",\"reason\": \"TN 4\", \"natural\": \"<=4\", \"highlight\": \"success invert\", \"countTotal\": \"<=4\"},\n{\"rolled\": \"D20\",\"reason\": \"TN 5\", \"natural\": \"<=5\", \"highlight\": \"success invert\", \"countTotal\": \"<=5\"},\n{\"rolled\": \"D20\",\"reason\": \"TN 6\", \"natural\": \"<=6\", \"highlight\": \"success invert\", \"countTotal\": \"<=6\"},\n{\"rolled\": \"D20\",\"reason\": \"TN 7\", \"natural\": \"<=7\", \"highlight\": \"success invert\", \"countTotal\": \"<=7\"},\n{\"rolled\": \"D20\",\"reason\": \"TN 8\", \"natural\": \"<=8\", \"highlight\": \"success invert\", \"countTotal\": \"<=8\"},\n{\"rolled\": \"D20\",\"reason\": \"TN 9\", \"natural\": \"<=9\", \"highlight\": \"success invert\", \"countTotal\": \"<=9\"},\n{\"rolled\": \"D20\",\"reason\": \"TN 10\", \"natural\": \"<=10\", \"highlight\": \"success invert\", \"countTotal\": \"<=10\"},\n{\"rolled\": \"D20\",\"reason\": \"TN 11\", \"natural\": \"<=11\", \"highlight\": \"success invert\", \"countTotal\": \"<=11\"},\n{\"rolled\": \"D20\",\"reason\": \"TN 12\", \"natural\": \"<=12\", \"highlight\": \"success invert\", \"countTotal\": \"<=12\"},\n{\"rolled\": \"D20\",\"reason\": \"TN 13\", \"natural\": \"<=13\", \"highlight\": \"success invert\", \"countTotal\": \"<=13\"},\n{\"rolled\": \"D20\",\"reason\": \"TN 14\", \"natural\": \"<=14\", \"highlight\": \"success invert\", \"countTotal\": \"<=14\"},\n{\"rolled\": \"D20\",\"reason\": \"TN 15\", \"natural\": \"<=15\", \"highlight\": \"success invert\", \"countTotal\": \"<=15\"},\n{\"rolled\": \"D20\",\"reason\": \"TN 16\", \"natural\": \"<=16\", \"highlight\": \"success invert\", \"countTotal\": \"<=16\"},\n{\"rolled\": \"D20\",\"reason\": \"TN 17\", \"natural\": \"<=17\", \"highlight\": \"success invert\", \"countTotal\": \"<=17\"},\n{\"rolled\": \"D20\",\"reason\": \"TN 18\", \"natural\": \"<=18\", \"highlight\": \"success invert\", \"countTotal\": \"<=18\"},\n{\"rolled\": \"D20\",\"reason\": \"TN 19\", \"natural\": \"<=19\", \"highlight\": \"success invert\", \"countTotal\": \"<=19\"},\n{\"rolled\": \"D20\", \"natural\": \"1\",\"highlight\": \"info invert bold\",\"content\":\"Add 1 extra success\"}\n]\n}"

Aug 31, 2024 12:31 am

Keleth says:

- Lets list out what data has/likely has emojis so I can fix all the fields

[ +- ] To the best of my imagination

Chalrytharendir

Aug 31, 2024 1:49 am

That's a great list!

Also character sheets

Last edited August 31, 2024 1:49 am

Chalrytharendir

Aug 31, 2024 1:50 am

Also, well done Keleth on solving the emoji problem! That was a tough one!

Aug 31, 2024 1:53 am

Chalrytharendir says:

Also, well done Keleth on solving the emoji problem! That was a tough one!

I have a solution, though I don't understand parts of it. For those curious, the sum is this:

Quote:

SELECT CONVERT(BINARY(CONVERT(title USING latin1)) USING utf8mb4) FROM forums;

Why latin1? Why does prod work, even though the script I created to just pull and display the prod data doesn't?

Aug 31, 2024 12:33 pm

Keleth says:

...

Quote:

SELECT CONVERT(BINARY(CONVERT(title USING latin1)) USING utf8mb4) FROM forums;

Why latin1? ...

My assumption when I suggested latin1 for the export/import process was that most data was entered by people/systems assuming ASCII as a baseline, so dropping all three byte character codes to being three separate bytes (even if not legit ACSII) instead of some broken utf8 combination, and then having the system interpret those three bytes back as utf8 might work. (Some of them might be smaller than three bytes, I don't claim any knowledge of what mysql is doing, I'm just preemptively casting Protection From Pedants. :)

latin1 because 'lowest common denominator' and all?

Keleth says:

... Why does prod work, even though the script I created to just pull and display the prod data doesn't?

That part does worry me. My concern is that you are fixing something for php that will require you to fix it again when you move away from php. But, since you have worked out how to fix it, that should not be too onerous?

The mysql command itself was showing the previously imported ones correctly while php was breaking them.
Now the mysql command is showing those that previously worked [ref] as broken.

I created a simple php8.2-cli container and it also prints the unicode as php7 does, so it is not an 'old php' issue.

Aug 31, 2024 1:10 pm

So python (mysql.connector) also renders the above query correctly, so we can probably ignore the fact that the official mysql command line client is doing something weird.

Aug 31, 2024 10:05 pm

FlyingSucculent says:

Bookmarks for games and characters get messed up on import.
Some private games are randomly marked public (but are not functionally public).

Can you provide examples of these so I can compare?

FlyingSucculent says:

Subscription are empty (I can't actually confirm if it's still true since I don't have any, but vagueGM reported it earlier.)
Advanced rules definitions display without line breaks, with \n instead.

These two are fixed and can be confirmed.

Sep 1, 2024 5:47 am

Keleth says:

Can you provide examples of these so I can compare?

Bookmarks
For games, I'm missing all of my bookmarks, there is not a single bookmark on staging at all. For a direct comparison, games 2100, 2211 and 4545 are bookmarked on production and aren't on staging.
(Edit: oops, copied wrong IDs, fixed.)

For characters, on production I have only two bookmarks of custom/13984 and custom/17142, while on staging I have the same two and additionally 13thage/16701.

Game status
The following games are all private on production and are marked as public on staging:
2100, 4545, 4092.
(There are way more - it's most of my games, actually.)

Additionally, game 4467 is public on production but is private on staging.
I think it might have just flipped all private and public ones to the opposite?

Keleth says:

These two are fixed and can be confirmed.

Can confirm that advanced rules are fixed.

Last edited September 1, 2024 6:02 am

Sep 1, 2024 8:51 am

My Game and Character bookmarks are nonsensical. What is displayed in the Games and Characters menu above does not reflect which games are marked in the Game and Character lists.

The marked lists are the same on both prod and staging, so the data came over, but the menu above does not reflect this data.

As shown in the below screenshot: Liv Diver is shown in the menu, but not marked in the list, Margery is marked but not shown.

Sep 1, 2024 8:57 am

The Latest Games block on the front page is empty. It should at least show the games listed in the Latest Public Games tests which were created yesterday? On prod is shows both Public and Private games, @FlyingSucculent could you make one of your Test games Private so we can see if it separates them that way? (My emoji test game is gone.)

Sep 1, 2024 9:00 am

My Subscription list only contains the Forums, not the Threads I have Subscribed to.

The Threads list is empty, the Forums list looks correct.

Sep 1, 2024 9:08 am