Data Download

This sample notebook demonstrates how to programatically download and visualize air quality data stored on the Open Storage Network

df_BME280 = CSV.File(download("https://ncsa.osn.xsede.org/ees230012-bucket01/AirQualityNetwork/data/raw/Central_Hub_1/2023/03/02/MINTS_001e06318c91_BME280_2023_03_02.csv")) |> DataFrame
965×5 DataFrame
940 rows omitted
Row dateTime temperature pressure humidity altitude
String31 Float64 Float64 Float64 Float64
1 2023-03-02 21:17:50.113837 24.88 96918.0 47.0 373.48
2 2023-03-02 21:18:00.131724 24.89 96913.0 47.0 373.91
3 2023-03-02 21:18:10.164866 24.88 96911.0 47.0 374.09
4 2023-03-02 21:18:20.181817 24.87 96914.0 47.0 373.83
5 2023-03-02 21:18:30.213784 24.88 96911.0 47.0 374.09
6 2023-03-02 21:18:40.229651 24.87 96914.0 47.0 373.83
7 2023-03-02 21:18:50.260480 24.88 96916.0 47.0 373.66
8 2023-03-02 21:19:00.276342 24.88 96910.0 47.0 374.17
9 2023-03-02 21:19:10.307070 24.88 96901.0 47.0 374.95
10 2023-03-02 21:19:20.322728 24.88 96901.0 47.0 374.95
11 2023-03-02 21:19:30.338601 24.88 96902.0 47.0 374.87
12 2023-03-02 21:19:40.369151 24.88 96903.0 47.0 374.78
13 2023-03-02 21:19:50.384637 24.89 96893.0 47.0 375.64
⋮ ⋮ ⋮ ⋮ ⋮ ⋮
954 2023-03-02 23:58:07.902762 27.38 96629.0 43.0 398.45
955 2023-03-02 23:58:17.919015 27.38 96621.0 43.0 399.15
956 2023-03-02 23:58:27.950124 27.38 96621.0 43.0 399.15
957 2023-03-02 23:58:37.981368 27.38 96627.0 43.0 398.63
958 2023-03-02 23:58:47.997385 27.37 96620.0 43.0 399.23
959 2023-03-02 23:58:58.028589 27.36 96624.0 43.0 398.88
960 2023-03-02 23:59:08.044753 27.36 96631.0 43.0 398.28
961 2023-03-02 23:59:18.075909 27.36 96619.0 43.0 399.32
962 2023-03-02 23:59:28.092216 27.36 96616.0 43.0 399.58
963 2023-03-02 23:59:38.123351 27.37 96610.0 43.0 400.1
964 2023-03-02 23:59:48.139750 27.37 96613.0 43.0 399.84
965 2023-03-02 23:59:58.170898 27.35 96622.0 43.0 399.06 
df_IPS7100 = CSV.File(download("https://ncsa.osn.xsede.org/ees230012-bucket01/AirQualityNetwork/data/raw/Central_Hub_1/2023/03/02/MINTS_001e06318c91_IPS7100_2023_03_02.csv")) |> DataFrame
9715×15 DataFrame
9690 rows omitted
Row dateTime pc0_1 pc0_3 pc0_5 pc1_0 pc2_5 pc5_0 pc10_0 pm0_1 pm0_3 pm0_5 pm1_0 pm2_5 pm5_0 pm10_0
String31 Int64 Int64 Int64 Int64 Int64 Int64 Int64 Float64 Float64 Float64 Float64 Float64 Float64 Float64
1 2023-03-02 21:18:03.686980 186348 101116 61137 6795 1036 10 0 0.155705 2.4369 8.82213 14.4999 28.0323 29.1087 29.1087
2 2023-03-02 21:18:04.756650 185877 100817 60893 6789 1036 10 0 0.155312 2.42976 8.78947 14.4623 28.0025 29.149 29.149
3 2023-03-02 21:18:05.682043 185455 100555 60617 6783 1040 16 0 0.154959 2.4235 8.75437 14.4225 28.0176 29.7319 29.7319
4 2023-03-02 21:18:06.679952 185011 100310 60232 6760 1041 17 0 0.154589 2.4176 8.70829 14.357 27.959 29.8246 29.8246
5 2023-03-02 21:18:07.677839 184486 99999 59783 6724 1042 17 0 0.154149 2.41013 8.65389 14.2727 27.8843 29.7314 29.7314
6 2023-03-02 21:18:08.674922 184024 99727 59374 6693 1041 18 0 0.153763 2.40361 8.60465 14.1975 27.7991 29.7457 29.7457
7 2023-03-02 21:18:09.672863 183592 99472 58951 6660 1038 19 0 0.153403 2.39749 8.55434 14.1197 27.6844 29.7084 29.7084
8 2023-03-02 21:18:10.670898 183114 99167 58509 6623 1035 16 0 0.153003 2.39022 8.50093 14.0356 27.5565 29.3077 29.3077
9 2023-03-02 21:18:11.667976 182595 98817 58055 6587 1032 10 0 0.152569 2.3819 8.44521 13.9493 27.4383 28.5185 28.5185
10 2023-03-02 21:18:12.666070 182069 98491 57559 6542 1028 7 0 0.15213 2.37411 8.38565 13.8521 27.2784 28.071 28.071
11 2023-03-02 21:18:13.663993 181435 98084 56997 6480 1013 5 0 0.1516 2.36438 8.31723 13.7318 26.9703 27.5012 27.5012
12 2023-03-02 21:18:14.661008 180749 97619 56413 6414 995 2 0 0.151027 2.35333 8.24519 13.6052 26.6081 26.8255 26.8255
13 2023-03-02 21:18:15.658345 180081 97165 55831 6350 973 0 0 0.150469 2.34253 8.17358 13.4801 26.198 26.2421 26.2421
⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮
9704 2023-03-02 23:59:48.539369 121501 59068 11955 2241 270 0 0 0.101522 1.43411 2.68273 4.55575 8.08815 8.08815 8.08815
9705 2023-03-02 23:59:49.537345 121060 58771 11910 2222 264 0 0 0.101154 1.42705 2.67102 4.52807 7.9793 7.9793 7.9793
9706 2023-03-02 23:59:50.535837 120637 58478 11872 2205 258 0 0 0.1008 1.42007 2.66001 4.50269 7.88151 7.88151 7.88151
9707 2023-03-02 23:59:51.534259 120133 58104 11802 2183 252 0 0 0.100379 1.41122 2.64386 4.46834 7.76922 7.76922 7.76922
9708 2023-03-02 23:59:52.532713 119602 57736 11719 2158 246 0 0 0.0999347 1.40247 2.62647 4.43038 7.64456 7.64456 7.64456
9709 2023-03-02 23:59:53.530269 119180 57461 11664 2140 240 0 0 0.0995824 1.39592 2.61415 4.4026 7.54067 7.54067 7.54067
9710 2023-03-02 23:59:54.528673 118754 57177 11616 2121 234 0 0 0.0992267 1.38914 2.60234 4.37504 7.43885 7.43885 7.43885
9711 2023-03-02 23:59:55.526219 118367 56925 11570 2102 229 0 0 0.0989031 1.38314 2.59152 4.34813 7.3423 7.3423 7.3423
9712 2023-03-02 23:59:56.524240 118005 56683 11505 2081 224 0 0 0.0986006 1.37738 2.57906 4.31805 7.24628 7.24628 7.24628
9713 2023-03-02 23:59:57.522713 117585 56398 11414 2056 218 0 0 0.0982495 1.37061 2.56273 4.28131 7.13573 7.13573 7.13573
9714 2023-03-02 23:59:58.520251 117207 56148 11337 2039 213 0 0 0.097934 1.36463 2.54869 4.25284 7.03624 7.03624 7.03624
9715 2023-03-02 23:59:59.518143 116909 55961 11277 2027 208 0 0 0.0976852 1.36018 2.53798 4.2321 6.95184 6.95184 6.95184

This is a sample markdown string

\[\begin{equation} \int_a^b f(x)dx \end{equation}\]

We can do equations!

Further, we can impute values. For example, the maximum PM 2.5 concentration for 5-2-2023 was 33.20207035 μg/m³

Let’s create a simple plot:

That plot looks great! Let’s now demonstrate the use of notebook parameters with papermill. In the first cell we define the variable test_parameter to the value 3.14. At execution time, the value is now 42