Ознакомьтесь с нашей политикой обработки персональных данных
  • ↓
  • ↑
  • ⇑
 
Записи с темой: bash (список заголовков)
16:28 

Converting files from another encoding while git pull

Vi Veri Veniversum Vivus Vici
  1. #!/bin/bash
  2. cd /home/user/proj
  3. FILES=()
  4. function testF
  5. {
  6. FILE=$1
  7. LEN=`wc -l $FILE | awk '{print $1}'`
  8. count=1;
  9. CHOMP=false
  10. if [ -s "$FILE" ] && [ "$(tail -c1 "$FILE"; echo x)" != $'\nx' ]; then
  11. LEN=($LEN+1)
  12. CHOMP=true
  13. fi
  14. while [[ $count -le $LEN ]]
  15. do
  16. LINE=`head -$count $FILE | tail -1`
  17. echo "$LINE" > /tmp/converter.txt
  18. isUTF=`file /tmp/converter.txt | grep UTF | wc -l`;
  19. if [ $isUTF != "1" ]; then
  20. iconv -f cp1251 -t utf8 /tmp/converter.txt -o /tmp/converter_1251.txt;
  21. mv /tmp/converter_1251.txt /tmp/converter.txt
  22. fi;
  23. cat /tmp/converter.txt >> "${file}.utf8"
  24. ((count++))
  25. done
  26. if [ $CHOMP == true ]; then
  27. perl -pe 'chomp if eof' "${file}.utf8" > "${file}.utf8.chomped"
  28. mv "${file}.utf8.chomped" "${file}.utf8"
  29. fi
  30. }
  31. git reset .
  32. git co .
  33. git clean -df
  34. git co .
  35. git merge --abort
  36. echo "NOW PULL"
  37. git stash
  38. git pull -Xtheirs origin master
  39. for i in `git status | grep "new file" | awk '{print $3}' | egrep "\.(php|js|htm|html|yml|xml|tpl)"`
  40. do
  41. FILES=("${FILES[@]}" "$i");
  42. done
  43. for i in `git status | grep "modified" | awk '{print $2}' | egrep "\.(php|js|htm|html|yml|xml|tpl)"`
  44. do
  45. FILES=("${FILES[@]}" "$i");
  46. done
  47. git stash pop
  48. echo "NOW CONVERT"
  49. for file in "${FILES[@]}"
  50. do
  51. DONE=false
  52. if [ -f "${file}" ]; then
  53. echo "FILE $file"
  54. testF "$file"
  55. echo "RENAME ${file}.utf8"
  56. mv "${file}.utf8" "$file"
  57. fi
  58. done

@темы: Bash

00:50 

MySQL concurrency with GET_LOCK

Vi Veri Veniversum Vivus Vici
I trying to use GET_LOCK for application-level blocking in my application. So I research how to do it in propert way. As I can see, IS_FREE_LOCK function is non-atomic, but GET_LOCK with second parameter (timeout) equal to 0 is works very good. So we should to use two checking steps: IS_FREE_LOCK as first and then GET_LOCK($lockId, 0) === 1.

First, we need to create code like this (its not ideal, and hasnt error handling: its a just proof-o-concept):

  1. <?php
  2. $db = new mysqli('localhost', 'root', '');
  3. if ($db->connect_errno) {
  4. printf("Не удалось подключиться: %s\n", $db->connect_error);
  5. exit();
  6. }
  7. function get_named_lock($lockname, $i) {
  8. global $db;
  9. $rs = $db->query("SELECT IS_FREE_LOCK('$lockname';) AS isfree");
  10. $result = $rs->fetch_array();
  11. if ($result['isfree']) {
  12. printf("Is free! PID = %s I = %d\n", getmypid(), $i);
  13. $rs = $db->query("SELECT GET_LOCK('$lockname', 0) AS locked");
  14. $result = $rs->fetch_array();
  15. $locked = $result['locked'];
  16. printf("%s Locked! PID = %s I = %d\n", $locked ? 'IS' : 'NOT', getmypid(), $i);
  17. return $locked;
  18. } else {
  19. return false;
  20. }
  21. }
  22. function release_named_lock($lockname) {
  23. $db->query("DO RELEASE_LOCK('$lockname';)");
  24. }
  25. printf("Started %s\n", getmypid());
  26. for ($i = 0; $i < 10000; $i++) {
  27. get_named_lock('mylock', $i);
  28. usleep(10);
  29. }
  30. printf("Release %s\n", getmypid());


Lets save it in file lock_test.php and then write small Bash sсript (run.sh):

  1. #!/bin/bash
  2. for i in {1..10}
  3. do
  4. `php lock_test.php >> /tmp/out.log` &
  5. done


Then you can to run it and see into logs:

$ ./run.sh
$ tail -f /tmp/out.log
Started 29376
Is free! PID = 29376 I = 0
IS Locked! PID = 29376 I = 0
Started 29369
Started 29377
Started 29368
Started 29370
Started 29373
Started 29374
Started 29372
Started 29378
Started 29379
Release 29373
Release 29370
Release 29376
Is free! PID = 29378 I = 9844
Is free! PID = 29369 I = 9964
Is free! PID = 29374 I = 9989
IS Locked! PID = 29369 I = 9964
Is free! PID = 29377 I = 9852
NOT Locked! PID = 29374 I = 9989
NOT Locked! PID = 29378 I = 9844
NOT Locked! PID = 29377 I = 9852
Release 29374
Release 29369
Release 29372
Is free! PID = 29377 I = 9876
Is free! PID = 29368 I = 9881
Is free! PID = 29379 I = 9906
Is free! PID = 29378 I = 9876
IS Locked! PID = 29379 I = 9906
NOT Locked! PID = 29378 I = 9876
NOT Locked! PID = 29368 I = 9881
NOT Locked! PID = 29377 I = 9876
Release 29379
Is free! PID = 29378 I = 9973
IS Locked! PID = 29378 I = 9973
Release 29368
Release 29378
Release 29377



As you can see, lot of sсripts can to catch lock as "free" at same time, but only one can actually catch it with GET_LOCK. Anyway, a lot of iterations are skipped by IS_FREE_LOCK check step - its fast and enough to skip non-concurrency scripts.

@темы: concurrency, PHP, MySQL, GET_LOCK, Bash, mutex

00:18 

Get last stable kernel

Vi Veri Veniversum Vivus Vici
curl -s www.kernel.org | grep -m 1 -o 'www.kernel.org/pub/linux/kernel/[^\"]*' > /tmp/last_kernel.txt && wget -v -d -i /tmp/last_kernel.txt && rm /tmp/last_kernel.txt

@темы: bash

Small Coder Blog

главная