Categories
notes tech

rust-pinger service status slack notifications

https://doc.rust-lang.org/std/thread/fn.sleep.html
https://doc.rust-lang.org/std/time/struct.Instant.html#method.now
https://rust-lang-nursery.github.io/rust-cookbook/file/read-write.html
https://doc.rust-lang.org/std/fs/struct.File.html#method.open
https://doc.rust-lang.org/std/io/trait.Read.html
https://docs.rs/chrono/latest/chrono/
https://stackoverflow.com/questions/27312069/how-can-i-iterate-over-a-vector-of-functions-and-call-each-of-them
https://doc.rust-lang.org/book/ch12-02-reading-a-file.html
https://stackoverflow.com/questions/26643688/how-do-i-split-a-string-in-rust
https://stackoverflow.com/questions/37888042/remove-single-trailing-newline-from-string-without-cloning
https://stackoverflow.com/questions/14154753/how-do-i-make-an-http-request-from-rust
https://docs.rs/reqwest/0.11.9/reqwest/
https://docs.rs/reqwest/0.11.9/reqwest/blocking/index.html
https://docs.rs/reqwest/0.11.9/reqwest/blocking/struct.Request.html
https://api.slack.com/apps/A03404BKQ1G/incoming-webhooks?success=1
https://doc.rust-lang.org/std/macro.format.html
https://docs.rs/toml/0.5.8/toml/
https://toml.io/en/
https://doc.rust-lang.org/std/net/struct.TcpStream.html

Categories
notes tech

[aws] docs

Need these for some part-time work

  • Launch Templates https://docs.aws.amazon.com/autoscaling/ec2/userguide/LaunchTemplates.html
  • Spot Fleet Scaling https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-fleet-step-scaling.html
  • EC2 Auto Scaling Limits https://docs.aws.amazon.com/autoscaling/ec2/userguide/asg-capacity-limits.html
  • EC2 Spot Instance Requests https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/spot-requests.html?icmpid=docs_ec2_console
  • Launch template –> AMI User data MIME scripts https://docs.aws.amazon.com/batch/latest/userguide/launch-templates.html#example-mount-an-existing-amazon-efs-file-system
  • Static (Elastic) IPs https://aws.amazon.com/premiumsupport/knowledge-center/ec2-associate-static-public-ip/
  • AWS EC2 autoscaling (external blog guide) https://www.cloudsavvyit.com/2043/getting-started-with-aws-autoscaling/
  • [StackOverflow] literally the point of Spot Instances https://stackoverflow.com/a/11996798/5945794
  • AWS SLA https://aws.amazon.com/compute/sla/
  • AWS EC2 SSH Connect https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-connect-methods.html

Pricings

  • EC2 On Demand https://aws.amazon.com/ec2/pricing/on-demand/
  • EC2 Spot Instance pricing history https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-spot-instances-history.html
  • EC2 Spot Pricing current history https://aws.amazon.com/ec2/spot/pricing/
  • Reserved EC2 instance https://aws.amazon.com/ec2/pricing/reserved-instances/pricing/
  • Cloudwatch https://aws.amazon.com/cloudwatch/pricing/
  • EC2 Autoscaling (no charge)
  • EBS Storage https://aws.amazon.com/ebs/pricing/

 

Categories
tech

[aws] intro to automated reasoning

https://www.amazon.science/blog/a-gentle-introduction-to-automated-reasoning

Categories
tech

[hn] docker optimisation guide (top 12)

Docker optimization guide: the 12 best tips to optimize Docker image security

 

Categories
tech

[hn] ssh-tools

https://news.ycombinator.com/item?id=30338289

https://github.com/vaporup/ssh-tools

Categories
notes tech

[scala] slack client api

https://github.com/slack-scala-client/slack-scala-client/issues

Categories
notes random tech

[hn[ front end redesigns are stupid

https://news.ycombinator.com/item?id=30382520

Categories
notes tech

[hn] block stackoverflow clone for DDG

https://github.com/quenhus/uBlock-Origin-dev-filter

Categories
tech

new mice [bought]

my mouse is shockingly bad at at higher sensitivity settings so cloud do with a new wirless mouse, currently logitech looking like the likely long term likely choice (anyone know of a synonym for choice that begins with the letter l?)

https://www.logitechg.com/en-gb/products/gaming-mice/g305-lightspeed-wireless-gaming-mouse.910-005283.html

https://www.logitechg.com/en-gb/products/gaming-mice/g403-hero-gaming-mouse.910-005633.html

https://www.logitechg.com/en-gb/products/gaming-mice/g604-lightspeed-wireless-gaming-mouse.910-005650.html


update — went for this one becuase (a) it’s cheaper (b) i’m not a cs:go professional so wireless isn’t the end of the world (c) it has a load of utility buttons so i can stop keyboard shortcutting everything.

now just have to hope it runs fine as PnP on linux.

Categories
notes tech

remarkable

Folks keep telling me I probably have some form of ADHD and the wealth of random pieces of paper that I scribbled on 14 months ago yet still occupy space in my living room may attest to that.

So this might be a worthy £400 expenditure to solve the “paper” problem without buying a full blown tablet.

https://remarkable.com

Categories
tech

atlas – database schema migration cli

https://atlasgo.io/

Categories
notes tech

auth0 jupyterhub authenticator

i’m sysadmin of some linux gpu servers in the phd working group and having to manually setup new users is a massive PITA.

a few options for replacements

  • Auth0 Authentication
  • KeyCloak OIDC / OAuth2 Authentication
    • https://www.keycloak.org/
    • functional+tested implementation up and running
    • only problem is existing user data — the user’s UID inside the spawned container depends on the order in which users start logging in, rather than ownership of local files.
    • could bind mount /home/{user} into container at /mnt/server then change $NB_UID for container user during usermod to match the UID of the user’s /home/{user}/.bashrc file — means another change to jupyterhub’s start.sh script in the Docker images.
    • Could also symlink /mnt/server to /home/{user}/server at end of start.sh with everything else in /home/{user} mounted in a docker volume? although docker volumes are NOT stored on the SSD for some servers because of disk space issue so user data wouldn’t be on the SSD :[
    • Better idea would probably be mount /home/{user} as before and change $NB_UID of $NB_USER to the uid of .bashrc (without chown-ing user files)
    • If no /home/{user} exists… then…. what happens? JupyterHub server does create the user account… so should be a simple case of look at uid of .bashrc … see it’s the same as $NB_UID… do nothing…?
    • BUT … JupyterHub will only create the /home/{user} directory inside its container (if using the completely containerised version) … which means that /home would still need to be mounted in the jupyterhub-{type} containers…. This will probably cause an absolute mess for user data though there will be multiple users that have files with UID=1000 etc. And because students can still SSH onto the servers it means they might be able to access other user’s data… :[
  • Native Authenticator
Categories
notes tech

sig-mlops

https://lists.cd.foundation/g/sig-mlops

This is a public list for the CDF MLOps SIG. All meetings and discussions are held in the open, and everyone is welcome to join. The current membership, calendar, and meeting documents can be found at https://github.com/cdfoundation/sig-mlops.

the Sig-MLOps 2021 roadmap speaks volumes to me and my experience as a machine learning phd student…

At this point in the development of the practice, it perhaps helps to understand that much of ML and AI research and development activity has been driven by Data Science rather than Computer Science teams. This specialisation has enabled great leaps in the ML field but at the same time means that a significant proportion of ML practitioners have never been exposed to the lessons of the past seventy years of managing software assets in commercial environments.

As we shall see, this can result in large conceptual gaps between what is involved in creating a viable proof of concept of a trained ML model on a Data Scientist’s laptop vs what it subsequently takes to be able to safely transition that asset into a commercial product in production environments. It is not therefore unfair to describe the current state of MLOps in 2020 as still on the early path towards maturity and to consider that much of the early challenge for adoption will be one of education and communication rather than purely technical refinements to tooling.

most other phd/professor type folks were not the least bit interested in dealing with these sorts of problems during development — basic codebase documentation often seems to be considered a waste of time based on the hundreds of github repos I’ve looked over.

it’s likely these are a subset of SEP fields and i’ve settled on calling the phenomena “Someone Else Will Clean Up the Mess” fields (or SEWCUM fields for short).

Categories
notes tech

“hacker laws”

https://github.com/dwmkerr/hacker-laws

the robustness principle is pretty apt for adversarial example research….

https://github.com/dwmkerr/hacker-laws#the-robustness-principle-postels-law