howpeter (31st March, 2014), pirsquared8 (27th April, 2013)
I've succeeded in getting Chinese EPG (specifically from SG Starhub) on OpenPLi 2.1 & enigma2, and I thought I share what I found and did. I'm a newbie on the scene so don't take what I say as the absolute truth. I'm still investigating and learning.
Side note: I understand cnigma2 (created by http://freedmx.net/?79) is a mod of enigma2 to support Chinese, but all I can find are .nfi binary images. Called me paranoid but I simply don't trust software from the scene without source code. (This is the reason I chose OpenPLi in the first place.)-- Part 1 --
Now, embedded in the DVB streams is something called DVB-SI (service information), which I read up @ http://www.dvb.org/technology/standa...468v1.12.1.pdf.
Side note: From section 5.2.5, you can see how your STB get its time. Using dvbsnoop (http://dvbsnoop.sourceforge.net/), I observe a TDT (time and date table) packet every 3 seconds.Using dvbsnoop (see http://dvbsnoop.sourceforge.net/exam...ample-eit.html for EIT (event info table) examples), for a Chinese programme, I see SH broadcast two short event descriptors (see section 6.2.37), one for English and one for Chinese. The ISO639 lang code for Chinese is "chi".
For non-English (or non-Latin lang), the "event_name_char" and "text_char" are to be encoded as specified in Annex A.
In Annex A.2, it says that if the first byte is less than 0x20, it denotes the encoding to be used, otherwise it is normal data.
SH violates this aspect. When the data is Chinese, it is sending it as UTF8 without putting a extra "0x15" byte (see table A.3) in front. Our STB, adhering to the spec, (including dvbsnoop) simply assumes it is normal data and uses Latin charset (ISO8859) to display it.
Example: The character "ju" is, as shown in http://www.scarfboy.com/coding/unicode-tool?s=E589A7 is encoded by three octets: 0xe5 0x89 0xa7. If the STB is aware they're UTF8, it will display these three octets as a single character; if it doesn't it will display these three octets as three separate characters, which is the case currently.
-- End of Part 1 --
Last edited by i1646tvk84423; 17th January, 2013 at 12:09 PM.
howpeter (31st March, 2014), pirsquared8 (27th April, 2013)
-- Part 2 --
As you know, enigma2 is the key app of our STB and controls many aspects including web server, interfacing with DVB drivers, etc. It is not the OS. The OS is GNU/Linux (which means it has a Linux kernel and GNU supporting utilities). enigma2 sits on top of this and is the main app.
It has a C++ core and a Python plugin framework.
As of around Oct 2011, DMM decides to close-source enigma2 (and also transition to enigma3 (closed source)?). DMM's enigma2 source is @ http://git.opendreambox.org/?p=enigma2.git;a=summary which you'll notice is absent of any .cpp source codes, only the include header files are available for Python plugin authors.
OpenPLi took the last available enigma2 source and fork it @ http://openpli.git.sourceforge.net/g...gma2;a=summary
Coming back to our objective, EITs are streaming continuously, so for this reason, enigma2 needs to use its C++ core to capture these into its EPG cache.
When an EIT is received, epgcache.cpp line 106 would lookup the ISO639 3-char encoding (function getCountryCodeDefaultMapping), while lines 112 & 113 calls a function called convertDVBUTF8.
getCountryCodeDefaultMapping is defined in encoding.cpp line 86. You can see the mapping is taken from a config files in /etc/enigma2/encoding.conf in your STB. Lines 39, 50, 53 and 61 shows the four available formats acceptable in encoding.conf.
convertDVBUTF8 is defined in estring.cpp at line 370. If SH follows the spec, line 423 would handle the extra 0x15 at the first, and send out the rest of the line as UTF8. Too bad, SH doesn't follow the DVB-SI spec.
Take a moment to analyze these 3 .cpp files, and you'll see that there is no way to support SH's Chinese EITs without modifying C++ code...
-- End of Part 2 --
howpeter (31st March, 2014), pirsquared8 (27th April, 2013)
-- Part 3 --
Since you need to modify code, now's a good time to start preparing your dev environment. Follow the steps @ http://users.telenet.be/spstefansp/openpliubuntu.html for a OpenPLi 2.1 dev environment. Don't try the 3.0, it has its own set of issues which I'm not ready to tackle right now.
Similar to http://thaidreambox.bayore.net/index.php?topic=15889, you need to add
between line 326 & 327 in UsageConfig.py. Now scp the new .py files to your STB:Code:("chi zho", _("Chinese")),
And delete the previous .pyo compiled version in your STB:Code:scp build-dm800se/tmp/sysroots/mipsel-oe-linux/usr/lib/enigma2/python/Components/UsageConfig.py root@DM800SE_IP:/usr/lib/enigma2/python/Components/
Restart enigma2 GUI to take effect.Code:rm /usr/lib/enigma2/python/Components/UsageConfig.pyo
Update your primary & secondary EPG language as follows:Code:kill `pidof enigma2`
Because SH sends English EIT for English programmes, but both Chinese and English EIT for Chinese programmes, you'll therefore want Chinese as the primary EPG language and English as the secondary language.
-- End of Part 3 --
Last edited by i1646tvk84423; 17th January, 2013 at 11:04 AM.
howpeter (31st March, 2014), pirsquared8 (5th August, 2013)
-- Part 4 --
Add the following two lines after line 373 in estring.cpp:
Recompile and a few enigma2 binaries will appear. Use "find" to find all of them. They should be of 2 sizes. Big 31MB ones that are not stripped and small 2.3MB ones that are stripped.Code:if (table == 99) return std::string((char*)data, len);
scp one of the 2.3MB stripped ones up to your STB:
If scp complains file is busy, it is because your enigma2 is running. /sbin/init 4 to kill it before you scp.Code:scp build-dm800se/tmp/work/dm800se-oe-linux/enigma2-2.7+git0+551e8f8bc26d672d0213065789239194258bd758-r36/package/usr/bin/enigma2 root@DM800SE_IP:/usr/bin/
Start or restart enigma2 to take effect.
Make your own encoding.conf:
and add one line:Code:cp /usr/share/enigma2/encoding.conf /etc/enigma2/
after line 9 in /etc/enigma2/encoding.confCode:chi ISO8859-99
-- End of Part 4 --
howpeter (31st March, 2014), pirsquared8 (27th April, 2013)
-- Part 5 --
Final piece of the puzzle.
The .ttf font /usr/share/fonts/nmsbd.ttf does not have Chinese, so you need to replace its use with a .ttf font that has Chinese. A candidate is simsun.ttf from MS.
Becareful. Your STB flash do not have much space left after OpenPLi, something like 16.9MB left. simsun.ttf is 14.6MB.
Then edit the skin definition:Code:scp simsun.ttf root@DM800SE_IP:/usr/share/fonts/
On line 15, changeCode:cd /usr/share/enigma2/PLi-HD/ vi skin.xml
toCode:<font name="Regular" filename="nmsbd.ttf" scale="100"></font>
Start or restart enigma2.Code:<font name="Regular" filename="simsun.ttf" scale="100"></font>
Voil?!
Side note: While the Chinese characters of simsun.ttf looks not bad, its English letters sux. Look at http://en.wikipedia.org/wiki/Unicode_fonts for alternatives.-- End --
howpeter (31st March, 2014), pirsquared8 (27th April, 2013)
Names of channels, specified in lamedb, can also be in Chinese UTF8. They will show when you're using a Unicode font.
Latest update: Microsoft's "Arial Unicode" looks the nicest, but it is more than 23MB big, way bigger than the ~ 17MB free space left after gjstroom's .nfi image. The next best looking font is "LiHei Pro", taken from my Mac OSX, which is what I've settled on. It is only 10MB, holds less glyphs than "Arial Unicode" but good enough for SH's Chinese so far..
Last edited by i1646tvk84423; 20th January, 2013 at 09:04 AM.
howpeter (31st March, 2014), pirsquared8 (27th April, 2013)
Omg, you're good.
one question why use chi zho?
is this hardcoded in the code for chinese?
("chi zho", _("Chinese")),
And can provide the recompiled files for the estring.cpp
Add the following two lines after line 373 in estring.cpp:
Code: Check this link
if (table == 99) return std::string((char*)data, len);
i do not have a linux environment to compile this. Thanks a lot!
Last edited by flye; 19th January, 2013 at 02:25 PM.
howpeter (31st March, 2014)
Normally, 1 language is one 3-letter code in ISO639, but some languages, including Chinese have 2. One is for simplified Chinese, the other for Traditional. The full list can be found at http://en.wikipedia.org/wiki/List_of_ISO_639-2_codes
SH uses only "chi" from what I can see. I put both codes just to be technically correct.
The same ISO639 codes are embedded in .mkv files that can contain multi-lang audio streams or subtitles streams.
1 concern is that version of your .nfi and modified engima2 binary must match. I got my .nfi latest from gjstroom ************* and my dev enironment pull down the latest source codes, so very high chance they will match.
My .nfi is dated 2013-01-15 and now, I see the latest gjstroom .nfi is already 2013-01-20.
I can put up my mod enigma2 binary later tonight, but as you can understand, no guarantees.
Ideally, you should have a dev enironment. If you can create a virtual linux machine, it will be easy.
Last edited by satsmo; 22nd January, 2013 at 11:02 PM. Reason: Removed link to forum of similar content
My enigma2 is at https://www.dropbox.com/s/q2aiiuvu84xheus/enigma2
My UsageConfig.py is at http://www.dropbox.com/s/qabfprzj9h3dsnu/UsageConfig.py
pirsquared8 (16th August, 2013)
dear sir, did anyone ever told you that you're a genius??
Oh man, you should have join earlier..... thanks !!!
time to dump cnigma and use my fav openpli !!
Last edited by flye; 22nd January, 2013 at 05:20 PM.
jatrophaseed (15th June, 2014), pirsquared8 (5th August, 2013)
Saw your attached pic. Well done!!
Glad my investigation help.
pirsquared8 (27th April, 2013)
"My enigma2 is at https://www.dropbox.com/s/q2aiiuvu84xheus/enigma2
My UsageConfig.py is at http://www.dropbox.com/s/qabfprzj9h3dsnu/UsageConfig.py"
Thanks for the explanation. As a non-coder. Is there a way for me to make those changes on my STB? (DM800HD-se)
Hopefully I can let this Chinese EPG issue be done and over with.
Thanks
Last edited by palmy; 25th January, 2013 at 05:42 AM.
Dude, great job man.
Could you get non-pli images, such as Newnigma2 to work as well?
A major reason I showed my steps is so that others can use this knowledge and apply it to any variants of enigma2 with source code.
As u can see above, the mod is just a 2-liner here and there. I'm 99.9% sure it will work on any variant of enigma2.
i1646tvk84423, previously with dm800 i was able to get FULL english channel list and epg. But now with dm800se, i'm getting a couple chinese channels with garbage programme listing.
What should I do if I want full english?
using newnigma 3.3.2
Bookmarks