Matrix tips they don't tell you

 

FAQ

 

1. Why is this in blogger.com? This is ugly

 

Migrated to fediverse! https://wordsmith.social/cos/matrix-tips-they-dont-tell-you

Mitä vikaa Telegramissa?

Telegram-pikaviestimestä on tullut melko suosittu joissain piireissä. Siinä on kuitenkin joitain puolia jonka vuoksi sitä ei voi suositella ainoana viestimienä uusien yhteisöjen perustamiseen.

Tässä on avattuna muutamia Telegramiin liittyviä ongelmia ja lopussa ehdotus ongelmien ratkaisemiseksi tai ainakin kiertämiseksi. Artikkeli on kirjoitettu lähinnä harrasteyhteisöjen näkökulmasta, yritysmaailmassa on eri kuviot. Lukijan oletetaan tietävän pikaviestinten eroavaisuudet tai osaavan selvittää ne tarvittaessa.

3. Tekniikka

 

Käyttäjän kannalta Telegramin asiakasohjelmat ovat samalla viivalla muiden nykyaikaisten viestimien kuten Slackin, Riotin, Mattermostin jne kanssa. Kaikissa on jonkinlainen kanavalista josta kanavan valitsemalla näkee keskustelun sisällön jossa virtaa tekstiä ja kuvia. Telegramin virallinen asiakasohjelma on avointa lähdekoodia josta iso peukku.

Telegramista puuttuu muihin palveluihin siltaamisen kannalta olennainen ominaisuus eli kummituskäyttäjät. Kummituskäyttäjillä (ghost user) tarkoitetaan sillan loihtimia käyttäjiä, jotka edustavat sillan toisessa päässä olevia käyttäjiä. Esimerkiksi IRC, Slack ja Matrix tukevat kummituksia ja niissä sillan toisessa päässä olevat käyttäjät näkyvät "oikeina" käyttäjinä.

Telegram-kanavan siltauksessa Telegramin päässä siltabotti toistaa toisten käyttäjien viestit mikä hieman vaikeuttaa keskustelun seuraamista ja yksityisviestien lähettäminen sillatuille käyttäjille on mahdotonta.

2. Yksityisyys

 

Telegramissa käyttäjän identiteettinä toimii puhelinnumero. Tämä kuulostaa alkuun ihan hyvältä idealta, mutta aiheuttaa ongelmia.

Yhdellä laitteella ei voi olla kuin yksi Telegram-tunnus. Et voi tehdä toista tunnusta esim työkäyttöön hankkimatta toista puhelinliittymää. Käyttäjänä näkyvän bottikäyttäjän luominen ei onnistu (bottien luomiseen on oma rajapintansa).

Puhelinnumero on erinomainen tapa yksilöidä ihminen tietokannassa. Yhdistämällä puhelinnumeron sisältämiä tietokantoja saadaan käyttäjästä kerätä paljon luotettavampaa tietoa joten puhelinnumerolla yksilöity tieto voidaan myydä kalliimmalla.

Telegram väittää että se ei myy käyttäjiensä tietoja kolmansille osapuolille. Tähän voi uskoa ne, jotka haluavat mutta sitä on vaikea todistaa suuntaan tai toiseen, koska palvelinpää on täysin suljettu.

1. Hallinto

 

Telegramin omistaa venäläisen Pavel Durovin rahoittama yritys Telegram Messenger LLP. Telegram pyörii täysin Durovin lahjoitusvaroin ja Durovin mukaan ei pyri tuottamaan voittoa.

Kuulostaa hyvältä? Mutta vaikka tilanne on tämä nyt, Telegram on kuitenkin yksittäisen tahon hallinnoima ja infrastruktuuriltaan suljettu palvelu. Jos Durov kääntää rahahanat kiinni, Facebook ostaa koko firman tai tapahtuu muuta yllättävää, voi tilanne muuttua eikä käyttäjillä ole siihen mitään valtaa.

Telegramin käyttäjillä ei ole mitään sanomista siihen mihin suuntaan se kehittyy ja mitä ominaisuuksia siihen tulee. Telegram sallii kolmannen osapuolen asiakasohjelmat, mutta sekin voi muuttua yksipuoleisella päätöksellä koska tahansa (muistatteko Facebookin XMPP:n ja Slackin IRC:n?).

Telegramin omistajan luotettavuutta arvioidessa kannattaa muistaa myös 1.7 miljardia dollaria tienannut kryptovaluutta-huijaus.

Mitä sitten kannattaa käyttää?

 

Jokainen yhteisö päättää miten haluaa palvella jäseniään. Nämä ovat omaan kokemukseeni perustuvia suosituksia. Oletuksena on, että yhteisö piittaa sekä tekniikasta, yksityisyydestä että hallinnosta. Jos näillä ei ole väliä, voi käyttää vaikka WhatsAppia (Älkää nyt oikeasti käyttäkö silti).

(Kuva ei mittakaavassa)

Olemassaolevat yhteisöt

Hacklab-yhteisöissä lähtötilanne oli se, että labit ja labien jäsenet käyttivät sekaisin IRCiä (kahdessa verkossa), Telegramia ja Slackia.

Noin 2018 tuli huhu uudesta viestimestä nimeltään Matrix millä pystyisi siltaamaan eri palvelut. Matrix otettiin koekäyttöön ja alkuaikoja vaivanneiden teknisten ongelmien jälkeen se on toiminut varsin mukavasti. Uusille suositellaan Matrixin käyttöä jolloin sillä pääsee "kaikille" kanaville helposti, mutta vanhat voivat käyttää omia suosikkiviestimiään kuten ennenkin.

Käyttäjän kannalta tälläisessä järjestelyssä ei ole väliä mitä viestintä kukakin käyttää, eikä se näy mitenkään keskustelussa poislukien Telegramin päässä oleva siltabotti. Uusia viestimiä voidaan ottaa kanavalle mukaan ja turhia poistaa tarpeen mukaan keskeyttämättä keskustelua.

WhatsApp on mahdollista sillata, mutta se on aikamoinen säätö

Harrasteyhteisöissä monilla on valtava muutosvastarinta pikaviestimien suhteen. "Ylhäältä päin" annettu määräys siirtyä käyttämään tiettyä viestintä ei toimi. Pahimmassa tapauksessa yhteisö jakaantuu kahteen koulukuntaan pelkästään sen takia että toinen puoli haluaa käyttää eri pikaviestintä. Matrixin sillat ovat olleet valtava apu sovun säilyttämisessä.

Uudet yhteisöt

Uusille yhteisöille kannattaa suosia samantien Matrixin käyttöä. Se on ei-teknisten ihmisten helppo ottaa käyttöön ja täyttää tekniset, yksityisyydelliset ja hallinnolliset vaatimukset paremmin kuin mikään muu. Tämän jälkeen kannattaa selvittää mitä viestimiä jäsenet mieluiten käyttävät ja pystyttää Matrixiin sillat k.o palveluihin, mikäli mahdollista.

Jos yhteisö on isompi ja teknisesti orientoitunut, kannattaa harkita oman Matrix-palvelimen pystyttämistä. Sillä käyttö on nopeampaa kuin matrix.orgin välillä ruuhkaisella palvelimella ja pystyttäminen ei ole valtavan vaikeaa.



t: @cos:hacklab.fi

Matrix Synapse authentication from Unix user accounts

Problem: You have an organization (university, business, hackerspace, etc) with existing user accounts on a Unix server. You want to host Synapse homeserver for them so that they can use their existing accounts to log in. 

Here's how to do it quite easily.

We use three main components:

• Synapse - the Matrix homeserver
• Synapse REST Password provider - This is a plugin to Synapse that acts as a HTTP client asking "I have client with this username and password, is it valid?" 
• pam-auth-rest-api - This is a HTTP server which receives the question, checks Unix PAM (Pluggable Authentication Modules) if the username / password pair is valid, and returns the information to the password provider.

You can run the homeserver on same or different server to where the accounts are. Security-wise it's probably better to have own server for Synapse.

Set up synapse

 

Read the installation instructions. I installed as Debian package. It doesn't matter how you install it. Remember to use postgresql database.

Use federation tester to make sure your homeserver is seen by the federation.

 

Set up REST Password provider

 

Read the instructions here. You just copy the plugin file to a specific directory. In homeserver.yaml you need to add the magic lines:

password_providers:
  - module: "rest_auth_provider.RestAuthProvider"
    config:
      endpoint: "http://server.with.accounts:3000"

 

Set up REST PAM API

This should be done on the computer with the Unix accounts, whether it is the homeserver or not.

Read instructions here.  You'll need to git clone the repository and do some setup. It is a node.js application so you'll need to install required nodejs packages as needed.

There is a systemd service file provided that can be used to automatically start the server.

After you have set up the server, test it with curl by running:

curl -X POST -d '{ "user": { "id": "@john.doe:example.org", "password": "passwordOfTheUser" } }' -H "Content-Type: application/json" http://localhost:3000/_matrix-internal/identity/v1/check_credentials  

This should return json with success: false (or true, if you provide real user credentials). 

You can also do the same test from Synapse server by replacing localhost with the hostname of the account server.

Test the setup

Now you can test the whole pipeline. Open a new private browser tab, load https://riot.im/app , Sign in, change homeserver to your homeserver and enter valid user account credentials. You should be logged in as a new Matrix user on the homeserver.

Note: homeserver name needs to be the real hostname of the server (https://matrix.example.org, not https://example.org).

If something fails, check the Synapse log and systemd log for the REST PAM API.

If you have questions or comment, I'm on #synapse:matrix.org and I'll try to help.


Good luck!

- @cosmo:modeemi.fi

End notes

• The REST Password provider is no longer maintained by its author. I'm tempted to fork and continue work on it unless someone else wants to.
• HTTP request between homeserver and account server is not encrypted. If these are different computers and there is possibility of a MITM attack, It's recommended to encrypt the traffic. I tried stunnel first (failed as the password provider doesn't like self-signed certs) and finally got it working with autossh ssh tunneling. You could also use reverse http proxy, vpn, or anything like that.

Gemini PDA hacker's guide

 Gemini PDA hacker's guide

Here's a collection of hints on how to do stuff on your Gemini they don't tell in the official documentation.

Linux warning

 Just a warning for non-hacker users: The Debian Linux for Gemini is still early alpha (12. Apr 2018) and it's not usable for daily work yet. Xorg uses slow software rendering, keyboard mapping is not right, Fn-modifier doesn't work so you can't type some characters etc. The best way to use it is from another computer via SSH.  This is why it's called technology preview.

Anyway, make sure you have everything backed up and ready to reflash everything if something breaks.

Flashing the operating system

There is now official Linux version of the flasher tool provided by Planet. Use it.

Accessing Linux via USB networking

If you are in situation when you can't access Linux side of Gemini, you can get there by USB networking:

1. Boot to Linux on Gemini
2. Connect USB cable
3. ssh gemini@10.15.19.82

 This works directly on Ubuntu.

If DHCP doesn't work, you can do sudo ip addr add 10.15.19.1/24 dev usb0; sudo ip link set usb0 up
(usb0 can be renamed into something else by udev, check dmesg)

Linux setup and general hints

I've created a shell script to help setting up Gemini's Debian:
https://github.com/gemian/gemini-scripts/blob/master/linuxsetup.sh

Usage is documented in the script itself, please read it.

NOTE:

• GDM display manager from Gnome doesn't seem to work. Gray screen after logging in. Use the default display manager.
• In Gnome the WiFi password dialog doesn't seem to work from top right corner menu. Open Settings / Network from menu, from there you can connect.
• By default Debian uses connman for managing network connections. You might want to remove it and install NetworkManager which is more commonly used by desktops. 
• Don't try to mount the android system partition from Linux, at least in rw  mode. Android won't boot after that and you'll need to reflash.
• To get larger widgets for touch usage in Gnome run:
  

  gsettings set org.gnome.desktop.interface scaling-factor 2 

Booting TWRP

It's possible to boot TWRP to install other distributions such as Sailfish on the Gemini. But booting it can be tricky. TWRP for Gemini can be found here.

1. Copy the files you want to flash with TWRP to Android Download folder, sd card or other place accessible from TRWP.
2. Boot to Debian.
3. Copy the TWRP .img file to Gemini from your pc:
1. scp twrp-geminipda-3.2.1-0.img gemini@10.15.19.82:
4. sudo dd if=twrp-geminipda-3.2.1-0.img of=/dev/disk/by-partlabel/recovery
5. Power off the device.
6. Power on the device by holding ESC for until screen turns on. TWRP should launch.

NOTE: Next time you boot Android, the recovery partition will be wiped and you need to repeat this!

Installing Sailfish

It's now in "closed beta". Don't ask me where to download it from. Hopefully it will be public soon. Here are the steps to install it in it's current form:

1. Install Android + Linux image, set up Debian (at least run resize2fs if you don't run the full script)
2. Make a backup of linux_boot: sudo dd of=linux-boot.img if=/dev/disk/by-partlabel/linux_boot
3. Download the sailfish .zip file to Android's Download directory. For some reason TWRP didn't find it if i just copied it to Android filesystem via Debian. MicroSD card should also work.
4. Boot TWRP.
5. Select the .zip from Downloads and install.
6. Power off and boot to Linux. Sailfish should start. The tutorial is a bit broken but you should get past it.
7. In Sailfish make a backup of linux_boot: dd of=sailfish-boot.img if=/dev/disk/by-partlabel/linux_boot

Switching between Sailfish and Debian

Overwrite the linux_boot partition with image you made earlier. For example to get from Sailfish to Debian, use dd if=linux-boot.img of=/dev/disk/by-partlabel/linux_boot

GUI for this would be nice.

NOTE: You can access Sailfish sysroot from Debian at /.stowaways/sailfishos and Linux sysroot from Sailfish by mounting /dev/mmcblk0p29 in any directory.

Why icons are needed in combat flight simulators

By cos / VLeLv Icebreakers

UPDATE 22.2.2017: Found 2 more sources for eye angular resolution, updated calculations based on it. 0.3 -> 0.6 arc-minutes.

In recent years it’s become a trend to force icons off in many multiplayer flight simulator arenas. The icons are said to be unrealistic and not needed for realistic simulation. As a real life pilot I have noticed that in real life you can see an aircraft several kilometers away and see it’s color, attitude and larger details. In flight simulators the same is not possible - you just see a pixel or few.

Seeing your opponent as soon as possible is paramount in air to air combat. I decided to study the issue and found out that icons must be used if you want to fly realistic air combat. Icons are a compromise caused by limitations in current display technology. You can fly without icons, but then the visual detection and identification ranges are much shorter than in real life.

Resolution comparison

Human eye is not a digital camera. It is analog device and has various smart features. The eyes move constantly and paint the image in our brain. Regardless of this, we can calculate a resolution for eyes for comparison.

According to [1], human eyes have FOV of about 200 degrees horizontal and 130-135 vertical.

There are several different values for the visual angle or one “pixel” in human eye.
According to [2] it's 0.3 arc-minutes. According to  [7] it's 0.6 arc-minutes.
According to [8] (page 16 table) it's 0.63.

Let's assume the correct value is 0.6 arc-minutes which is 0.01 degrees. If we limit our study to a more moderate usable FOV of 120 degrees in both directions, the human eye resolution is 144 megapixels [3].

A average modern monitor has resolution of 2560*1200 pixels. This is about 3 megapixels. The FOV is usually about 75 degrees in a flight simulator.

We can calculate how many pixels we would need in a monitor to match the eye angular resolution for 75 degree image: 7500*7500. That would be 56 megapixels.

One more geometry exercise: How many “pixels” wide is an aircraft with wingspan of 15m at 10km distance?

On a 2560*1200 monitor with 75 degrees horizontal FOV one pixel represents 0.0293 degrees [4].

The aircraft is 0.0859 degrees wide. This gives 8.6 “pixels” with human eye [5]. You should be able to see its orientation and larger details quite well.

With a monitor, this would give about 2.9 pixels [6]. You might be able to see aircraft color, but not orientation or other details. Even noticing the aircraft against terrain would be difficult.

In summary:

	Human eye	Typical monitor
Horizontal FOV (degrees)	200 (120 used in calculations)	75
Resolution (pixels)	7500*7500	2560*1200
Megapixels	144	3
Angle of one pixel (degrees)	0.01	0.0293
Size of 15m object at 10km (pixels)	8.6	2.9

With first generation VR headsets (OR CV1, HTC Vive, etc) the resolution issue is even worse - they typically have resolutions of about 1200*1200 which is less than half of the typical monitor used here. FOV is luckily better (110 degrees on both).

Peripheral motion detection

In addition to resolution one important factor is the peripheral motion detection. Eyes and brain are able to detect targets moving against background outside the high-resolution central vision area. When flying, this is very important as it allows noticing other aircraft  when looking elsewhere.

It's difficult to measure any significant numbers on the motion detection so it's excluded in this study. I also didn't find any data on if or how well this works on monitor compared to real life. At least the FOV is significantly smaller on monitor which naturally has an effect on target detection.

Suggestions for simulator developers

As the chapters above explains, icons are mandatory for realistic air combat in visual range. Many simulators implement icons in sub-standard way, and they often either unrealistic (too long range / visible through cockpit) and ugly (line of text).

Here’s my suggestion:

Use these ranges as baseline. Make sure that in multiplayer situation all players have the same settings forced. Players should have option to turn off icons, if they want.

Range	Icon
5km	Detect small aircraft
3km	Show aircraft type and friend or foe
1km	Show any other details as needed (weapon loadout, etc)

• Make the icons look good. Use alpha blending to gradually display them.
• Darkness, clouds, fog and sun glare should decrease the distances.
• Icon must not be shown if the target is behind obstacle (mountain) or in other way not visible from cockpit.
• Use visual means to display distance (a bar or arc instead of text). Distance is difficult to judge visually so this doesn’t have to be very accurate.
• Keeping the target in view for longer period can increase the distances. For example you might be able to detect a target or identify it's type if you look at it for several seconds.
• It is more difficult to notice targets against ground than against sky. Reduce the distances, if the target background is ground.
• Large aircraft should be detected and identified from longer distance.

For an example of well working icons, take a look at World War 2 Online. See videos of the simulator to see how the icons work.

Thanks for reading!

[1] https://en.wikipedia.org/wiki/Human_eye#Field_of_view

[2] http://www.clarkvision.com/articles/eye-resolution.html

[3] (120 / 0.01)^2

[4] 75 / 2560

[5] 0.0859 / 0.01

[6] 0.0859 / 0.0293

[7] https://en.wikipedia.org/wiki/Visual_acuity#Physiology

[8] http://www.icoph.org/dynamic/attachments/resources/icovisualacuity1984.pdf

Running Qt applications in Android

Digia (owner of Qt) is working on official Android and iOS ports of Qt, but meanwhile Qt applications can be built and run in Android using Necessitas, an open source project which forks official Qt SDK. Currently it is still in beta phase but is already usable. This document has been written in early February 2013 so it may not be valid with later Necessitas versions.

Setting up

 Necessitas SDK

Download and install Necessitas SDK from http://necessitas.kde.org/necessitas/necessitas_sdk_installer.php

Launch SDKMaintenanceTool from Necessitas. Select the desired API level from Miscellaneous/Android NDK. You could here also install Android SDK under Necessitas if you don't already have it installed.
Note: While testing API level 10 seemed to work the best. Later versions launched always in AVD even when a device was connected.

Android SDK

Download and install Android SDK from http://developer.android.com/sdk/index.html
Load the Android SDK version you want to support using android_sdk/tools/android tool.

Building

Launch Qt Creator from Necessitas installation using QtCreator/bin/necessitas (not bin/qtcreator!)

Open a project (existing one or create a new). A "Configure Project" dialog is shown.

Set the project as "Necessitas Qt 4.8.2 for Android armv7a" project. You can also select other platforms you might want to build to.

Select from menu Tools | Options and Android page. Set "Android SDK location" to the path you installed Android SDK. If you need to create a Android Virtual Device (emulator) create it now.

Some items were not autoconfigured in my install, so:

Set x86 GDB to android-ndk/toolchains/x86-4.4.3/prebuilt/linux-x86/bin/i686-linux-android-gdb
Set x86 gdbserver to android-ndk/prebuilt/android-x86/gdbserver/gdbserver
Set OpenJDK location to /usr/bin/java

 Click "Ok" to close dialog.
Click on build configuration icon in bottom left corner, above "Run" button. Select the Necessitas build and Debug or Release, which one you prefer.
Click Projects (on the left side bar) -> Run Settings -> Deployment -> Package configurations -> Android target SDK. Change it to the android version you plan to use.
In "Deploy configurations" you can select wheter to include Qt libraries in your package or use Ministro to download libraries on demand. Ministro seemed to work on device but not in AVD. Ministro is a "library downloader" which will install the required Qt libraries to Android so that they can be shared between Qt applications.

Press Ctrl-b to build the application.

Running

Press Ctrl-r to run the application. Application should start in device (if connected) or in AVD.
Sometimes i got error "Found some build errors in current task. Do you want to ignore them?" but ignoring errors worked and the application would run nicely.

Conclusion

The Necessitas SDK is clearly still beta quality, but already can be used to port existing Qt applications to Android or create Android apps from scratch. Source changes are normally not needed at all. Naturally UI's may need adjustment for touch interface if they have been implemented on desktop. Applications built
using Necessitas can be and have been published in Google Play store.

State of Linux video editors

I need to edit some videos randomly. Nothing professional, mostly some clips from a GoPro or cell phone video camera. I need to cut & paste clips and add music soundtrack. This should be the basic feature set of any video editor, but unfortunately most of them fail one way or another.

See also State of Linux audio players

Here is a comparison of few free video editors available. They are tested on Ubuntu.

Kdenlive (tested march 2014)

Current winner. Does all I need without issues. UI could use some polishing, but good enough.

OpenShot (tested summer 2013)

Decent video player. Crashes randomly but is still usable. Ui misses some features, such as moving a clip to end of previous clip which makes editing long videos painful. Also there's still no way to select multiple clips.

PiTiVi 0.15 (tested jan 2012)

Mediocre but not bad UI. Preview becomes so slow that it's practically a slideshow with multiple 720p videos. Crashes frequently. Development is active but work is done for next version based on gstreamer editing services. I had a problem with rendered files having zero size, but this was fixed by removing gaps from the project. Many thanks to thiblahute on #pitivi.

Cinelerra (tested spring 2011)

Crashes so often that it's useless. UI looks like it's from 90's.

VLMC (checked summer 2013)

Spawned from VLC media player. Looks promising but i suppose develoment is dead as the official repository doesn't have packages for current or previous Ubuntu release.