Points regarding computing that I always forget - so I write them here to find them easily and maybe other could benefit too.
Thursday, November 30, 2023
Tuesday, November 28, 2023
List of software and apps that does not require registration and just works
List of software and apps that does not require registration and just works out of the box without too many ads and without need to registrations. The list of applications for Android and Linux desktop that just do the work they need to do. Nothing more. Nothing less.
- K-9 Email Mail for Android
- Otomusic - music mp3 player application for Android
Wednesday, November 15, 2023
Writing a json to smtp email proxy in go (golang) json2smtp
I wrote a json2smtp proxy server that can be self hosted in Go (golang)
See here:
https://www.c2kb.com/json2smtp
https://github.com/caviv/json2smtp
The reason why I needed to write an email proxy was because I was called to do some maintenance on an old legacy system. This system was running on Windows Server 2012, with php version 5.5.6 and Apache 2.4. Very old versions of software. The system was sending emails using the php smtp package and old libssl.dll which supports only TLSv1.0. The smtp service they were using (AWS SES Simple Email Service) has recently dropped the support for SMTP connection via TLS version 1.0. Now in order to support TLS v1.2 for emails on smtp I have to upgrade the whole system. Windows version, php version, Apache version and more - this would have taken me a very long time with many obstacles on the way.
Exim is another popular open-source mail server that can be used as an email proxy. It can be configured to receive emails in JSON format and send emails through its own SMTP servers. | might be ok but very hard to configure.
Mailcow Dockerized is a Docker image that contains a number of email server components, including Postfix, Dovecot, and MySQL. It can be configured to receive emails in JSON format and send emails through its own SMTP servers. | Couldn't make it work
An email proxy: input: json, output: smtp call
For a legacy project I needed to have a proxy email that reads json input and execute a smtp calls in order to send emails. So I created a small proxy for emails in go (golang)
See the code on github: https://github.com/caviv/json2smtp
Download json2smtp binaries executables
Ubuntu linux: json2smtp
Windows amd64: json2smtp-amd64.exe
How it works:
Simple calling diagram:
The json struct object
Simple object:
curl -X POST \
-H "Content-Type: application/json" \
-d '{ \
"from": "john doe <john@example.com&rt;", \
"to": ["kermit@muppets.com", "oneperson@example.com"], \
"cc": ["email1@example.com"], \
"bcc": ["secret@example.com"], \
"subject": "email subject line", \
"message": "message body in text/html to be sent", \
"attachments": {"filename.pdf": "base64 file encoded",
"anotherfilename.txt": "base64 file encoded"}, \
}' \
http://localhost:8080/
Full object with smtp data:
curl -X POST \
-H "Content-Type: application/json" \
-d '{ \
"from": "john doe <john@example.com&rt;", \
"to": ["kermit@muppets.com", "oneperson@ex", \
"to": ["kermit@muppets.com", "oneperson@example.com"], \
"cc": ["email1@example.com"], \
"bcc": ["secret@example.com"], \
"subject": "email subject line", \
"message": "message body in text/html to be sent", \
"attachments": {"filename.pdf": "base64 file encoded",
"anotherfilename.txt": "base64 file encoded"}, \
"smtphost": "smtp.example.com - optional parameter", \
"smtpport": 587 - optional paramater, \
"smtpuser": "username - optional parameter", \
"smtppassword": "password - optional parameter" \
}' \
http://localhost:8080/
Attachments
In order to send attachments with your json email struct you need to construct an object of base64 encoded string of your binary file.How to install email proxy server:
Download the code and run it
git clone https://github.com/caviv/json2smtp.git
go run ./
go run ./ --help
Build and compile
Download the code compile it and run with help command
git clone https://github.com/caviv/json2smtp.git
go build ./
./json2smtp --help
Execute smtp2json emal proxy and samples
Command line help:
json2smtp utility https://www.c2kb.com/json2smtp v1.0.1 2023-11-13
Get json input and calls smtp - function as a json to smtp proxy
Options:
-port int
the port to listen on (default 8080)
-smtphost string
smtp host, e.g. smtp.example.com
-smtpoverride
true - allows to pass smtp parameters in the json call,
false will always use the config smtp data (default true)
-smtppassword string
password for the smtp user
-smtpport int
the port to listen on (default 587)
-smtpuser string
username for the smtp
Example:
json2smtp --port=8200 --smtphost='smtp.example.com'
--smtpport=587 --smtpuser='username'
--smtppassword='password'
--smtpoverride=false
Run service in the background:
nohup json2smtp --port=8200 --smtphost='smtp.example.com'
--smtpport=587 --smtpuser='username'
--smtppassword='password' >> logfile.log 2>&1 &
Simple execute smtp2json service:
In this way the calling client will have to pass the smtp server details in each call because we don't set the smtp default server for the proxy. The default port to listen on is 8080.
json2smtp
Recommended architecture to run the service:
Libraries used
This external libraries are used in the project:
require gopkg.in/mail.v2 v2.3.1
require gopkg.in/alexcesaro/quotedprintable.v3 v3.0.0
Sunday, November 5, 2023
Taking a screenshot and creating thumbnail of webpage html as an image via command line
There are a few options to take screenshot of a webpage or save a webpage as an image on the disk via the command line. This can allow you to create a thumbnail of a web site easily by one command on the terminal.
Using chrome to save a webpage as an image:
chrome --screenshot=google.png --headless --window-size=1024,768 https://www.google.com/
Using cutycapt to screenshot a webpage:
https://cutycapt.sourceforge.net/
Using webkit2png to create site thumbnail:
https://paulhammond.org/webkit2png
You can also use Midori browser
Tuesday, October 17, 2023
Linux Ubuntu Installing and Using Fake Camera Stream
If you wish to create a virtual camera in Linux and stream "fake" videos during calls:
To install the loopback camera:
sudo apt update
sudo apt install v4l2loopback-dkms v4l2loopback-utils
To add the loopback camera module:
sudo modprobe v4l2loopback card_label="My Fake Webcam" exclusive_caps=1
or
sudo modprobe v4l2loopback devices=1 video_nr=1 card_label='MyWebCam' exclusive_caps=1
Now to stream the video:
First find the list of videos device available:
v4l2-ctl --list-devices
You can check the videos by playing:
ffplay /dev/videoX
ffplay Rick_Astley_Never_Gonna_Give_You_Up.mpg
Now you can stream the video to the free one:
ffmpeg -re -i Rick_Astley_Never_Gonna_Give_You_Up.mpg -map 0:v -f v4l2 /dev/videoX
Or
ffmpeg -stream_loop -1 -re -i video.mp4 -vcodec rawvideo -threads 0 -f v4l2 /dev/videoX
To remove the module from the kernel:
sudo modprobe --remove v4l2loopback
Software for streaming:
sudo apt-add-repository ppa:obsproject/obs-studio
sudo apt-get update
sudo apt-get install ffmpeg obs-studio
Monday, July 31, 2023
Go (golang) howto not to export fields of a struct when Unmarshal to json object
Use:
typedef MyData struct {
Data string `json:"data"`
FieldName *int64 `json:"-"`
}
Adding the `json:"-"` will make the Unmarshal command disregard this field
Saturday, July 22, 2023
Cleaning and resizing VM VirtualBox VDI Virtual Machine Disk
You have to do the following steps:
Run defrag in the guest (Windows only)
Nullify free space:
With a Linux Guest run this:
dd if=/dev/zero of=/var/tmp/bigemptyfile bs=4096k ; rm /var/tmp/bigemptyfile
Or:
telinit 1
mount -o remount,ro /dev/sda1
zerofree -v /dev/sda1
With a Windows Guest, download SDelete from Sysinternals and run this:
sdelete.exe c: -z
(replace C: with the drive letter of the VDI)
Shutdown the guest VM
Now run VBoxManage's modifymedium command with the --compact option:
With a Linux Host run this:
vboxmanage modifymedium --compact /path/to/thedisk.vdi
With a Windows Host run this:
VBoxManage.exe modifymedium --compact c:\path\to\thedisk.vdi
With a Mac Host run this:
VBoxManage modifymedium --compact /path/to/thedisk.vdi
This reduces the vdi size.
https://superuser.com/questions/529149/how-to-compact-virtualboxs-vdi-file-size
Thursday, May 25, 2023
Some notes about caddy webserver installation on Linux
caddy
https://caddyserver.com/docs/install#debian-ubuntu-raspbian
sudo apt update
sudo apt install -y debian-keyring debian-archive-keyring apt-transport-https
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/gpg.key' | sudo tee /etc/apt/trusted.gpg.d/caddy-stable.asc
curl -1sLf 'https://dl.cloudsmith.io/public/caddy/stable/debian.deb.txt' | sudo tee /etc/apt/sources.list.d/caddy-stable.list
sudo apt update
sudo apt install caddy
# the code sits in /var/www
# create link to the local directory
ln -s /var/www/ /home/xxx/myprojects/web
# set the user to be xxx and the group caddy
sudo chown -R xxx:caddy ./www
# add us to caddy groups
sudo usermod -a -G caddy xxx
sudo usermod -a -G www-data xxx
Install php-fpm
sudo apt install php-cli php-fpm php-mysql php-mbstring php-curl
https://www.howtoforge.com/tutorial/ubuntu-caddy-web-server-installation/
configure caddy with php
sudo nano /etc/php/8.1/fpm/pool.d/www.conf
#change users to caddy (user, group, listen.owner, listen.group)
#take the socket name: listen = /run/php/php8.1-fpm.sock
nano php.ini # set E_ALL view all errors
sudo systemctl restart php8.1-fpm
#set socket in Caddyfile: php_fastcgi unix//run/php/php8.1-fpm.sock
sudo service caddy stop && sudo service caddy start
Caddyfile
website.com {
#:80 {
tls /var/www/fullchain.pem /var/www/privkey.pem
# Set this path to your site's directory.
root * /var/www/website.com
php_fastcgi unix//run/php/php7.4-fpm.sock
# Enable the static file server.
# file_server
rewrite /symbol/* s.php?s={path}
# rewrite /symbol/ {
# r ^/(\w+)/?$
# to /symbol.php?symbol={1}
# }
file_server
encode gzip
log {
output file /var/log/caddy/example.com.access.log {
roll_size 3MiB
roll_keep 5
roll_keep_for 48h
}
format console
}
# Another common task is to set up a reverse proxy:
# reverse_proxy localhost:8080
# Or serve a PHP site through php-fpm:
# php_fastcgi localhost:9000
#RewriteRule ^symbol/jscss/([^/]*)\.([^/]*)$ /jscss/$1.$2 [L] # no need for those since we used <base href="../" />
#RewriteRule ^symbol/images/([^/]*)\.([^/]*)$ /images/$1.$2 [L]
#rewrite /symbol/* / symbol.php?symbol={query}
}
I had the same problem whith my docker php-fpm and I fixed it by modifing parameters
in these files: /usr/local/etc/php-fpm.d/www.conf /usr/local/etc/php-fpm.d/www.conf.default
The new parameters are :
pm = dynamic
pm.max_children = 25
pm.start_servers = 10
pm.min_spare_servers = 5
pm.max_spare_servers = 20
pm.max_requests = 500
- ADD how to enabled logs in PHP
Monday, February 6, 2023
Linux terminal shell command line to show a list of just the files in the directory
Linux terminal shell command line to show a list of just the files in the directory
find . -type f
Wednesday, February 1, 2023
php8 COM Exception Type Mistmtach in paramater #0 when calling MS Word COM object SaveAs
php8 COM Exception Type Mistmtach in paramater #0 when calling MS Word COM object SaveAs
I saw this:
https://bugs.php.net/bug.php?id=73605
When calling $word->SaveAs($filename, 8);
You should change to $word->SaveAs($filename, new variant(8, VT_I4);
php 8 using ldap connectivity configuring php.ini and apache2.4
Got an error that ldap_connect is not a recognize function to use with apache 2.4, php 8 on windows
Here is a solution:
1.
Enable extension=ldap in the php.ini
2.
Copy .dll files from the php installation directory into the apache/bin installation directory
Please note, ldap_sort function was deprecated
Monday, January 16, 2023
Linux Ubuntu Changing the datair of mysql
mysqld.cnf
datadir = /new/data/dir
mysqld --user=mysql --initialize
innodb_flush_log_at_trx_commit= 0
ssl=0 # (5.7 community (self built) might not use SSL for cases. 8.0 always use ssl by default, might be slower)
innodb_doublewrite_pages=120 # (if doublewrite enabled for recent 8.0.x "on HDD")
innodb_undo_log_truncate=OFF
innodb_log_writer_threads=OFF
plugin-load-add=auth_socket.so
auth_socket=FORCE_PLUS_PERMANENT
sudo systemctl stop apparmor
sudo systemctl disable apparmor
sudo mysqldump --master-data=1 --flush-logs --single-transaction --routines --quick --all-databases > db20230116_1852.sql
CREATE USER 'XXXXX'@'%' IDENTIFIED WITH mysql_native_password BY 'XXXXXX';
GRANT ALL PRIVILEGES ON *.* TO 'XXXXX'@'%' WITH GRANT OPTION;
FLUSH PRIVILIGES;
Saturday, January 14, 2023
mysql linux client get stuck on command line connecting mysql server 8.0
use
mysql --ssl-mode=DISABLED -p