Python/Building a Real-Time System Monitoring Dashboard with Python.md
... ...
@@ -0,0 +1,605 @@
1
+https://towardsdev.com/building-a-real-time-system-monitoring-dashboard-with-python-6e09ff15e0ff
2
+
3
+# Building a Real-Time System Monitoring Dashboard with Python | by Py-Core Python Programming | Towards Dev
4
+[
5
+
6
+![Py-Core Python Programming](https://miro.medium.com/v2/resize:fill:88:88/1*rLxk1HNyO_Pq_EFVFAzNbw.png)
7
+
8
+
9
+
10
+](https://medium.com/@ccpythonprogramming?source=post_page---byline--6e09ff15e0ff--------------------------------)
11
+
12
+[
13
+
14
+![Towards Dev](https://miro.medium.com/v2/resize:fill:48:48/1*c2OaLMtxURd1SJZOGHALWA.png)
15
+
16
+
17
+
18
+](https://towardsdev.com/?source=post_page---byline--6e09ff15e0ff--------------------------------)
19
+
20
+System Dashboard with Plotly in Python
21
+
22
+Let’s create a dashboard using Python that monitors your system’s performance in real-time. The dashboard will display your PC’s RAM, CPU load and disk space usage.
23
+
24
+We will use **Dash** for the web interface, **Plotly** for interactive graphs, **psutil** for system monitoring.
25
+
26
+This will work on both Windows and Linux systems. However, for macOS users or special configurations, slight modifications may be needed.
27
+
28
+![](https://miro.medium.com/v2/resize:fit:720/format:webp/1*feYT5wJ6FmdIp42BxaHT_A.gif)
29
+Dash/**Plotly** Python GIF
30
+
31
+Project Setup
32
+-------------
33
+
34
+Step 1: Create a Project Folder
35
+-------------------------------
36
+
37
+Begin by creating a folder for the project. Open a terminal (or command prompt) and create a new directory.
38
+
39
+```
40
+mkdir system_monitor_dashboard
41
+cd system_monitor_dashboard
42
+```
43
+
44
+
45
+Step 2: Set Up a Virtual Environment
46
+------------------------------------
47
+
48
+Isolating your project dependencies is important. We’ll create a virtual environment to keep our environment clean and predictable.
49
+
50
+For Windows:
51
+------------
52
+
53
+```
54
+python -m venv venv
55
+venv\Scripts\activate
56
+```
57
+
58
+
59
+For Linux/Mac:
60
+--------------
61
+
62
+```
63
+python3 -m venv venv
64
+source venv/bin/activate
65
+```
66
+
67
+
68
+Step 3: Install Required Libraries
69
+----------------------------------
70
+
71
+We need to install the following Python packages:
72
+
73
+* `dash`: Web framework for building dashboards.
74
+* `plotly`: Used to create graphs and charts.
75
+* `pandas`: For handling data in a tabular format.
76
+* `psutil`: For monitoring system resources.
77
+
78
+Install them by running:
79
+
80
+```
81
+pip install dash plotly pandas psutil
82
+```
83
+
84
+
85
+This will install all the libraries required to build our dashboard.
86
+
87
+Building the Dashboard
88
+----------------------
89
+
90
+Step 4: Writing the Code
91
+------------------------
92
+
93
+Now we’ll create a Python script that gathers system statistics and displays them using a dynamic, real-time dashboard.
94
+
95
+1. **Monitor RAM, CPU, and Disk**
96
+ The `psutil` library provides cross-platform access to system monitoring information. It works on Windows, Linux, and macOS.
97
+
98
+Create a new Python file called `app.py`:
99
+
100
+```
101
+touch app.py
102
+```
103
+
104
+
105
+Open `app.py` and paste the following code:
106
+
107
+```
108
+import dash
109
+from dash import dcc, html
110
+from dash.dependencies import Input, Output
111
+import plotly.graph_objs as go
112
+import psutil
113
+import logging
114
+from collections import deque
115
+from datetime import datetime
116
+import sys
117
+# Set up basic logging to debug
118
+logging.basicConfig(level=logging.INFO)
119
+# Initialize the Dash app
120
+app = dash.Dash(__name__)
121
+# Define fixed-size lists (deque) to store the last 20 data points for RAM, CPU, Disk usage, and time
122
+history = {
123
+ 'ram': deque(maxlen=20),
124
+ 'cpu': deque(maxlen=20),
125
+ 'disk': deque(maxlen=20),
126
+ 'time': deque(maxlen=20) # Store timestamps for x-axis
127
+}
128
+# Function to get system statistics (RAM, CPU, and Disk)
129
+def get_system_stats():
130
+ try:
131
+ # Get memory stats
132
+ memory = psutil.virtual_memory()
133
+ ram = memory.percent
134
+ # Get CPU usage
135
+ cpu = psutil.cpu_percent(interval=1)
136
+ # Get Disk usage
137
+ disk = psutil.disk_usage('/').percent
138
+ # Return RAM, CPU, and Disk data
139
+ return {
140
+ 'RAM Usage (%)': ram,
141
+ 'CPU Usage (%)': cpu,
142
+ 'Disk Usage (%)': disk
143
+ }
144
+ except Exception as e:
145
+ logging.error(f"Error fetching system stats: {e}")
146
+ return {}
147
+# Determine whether to run in 'one' or 'multiple' mode based on command-line argument
148
+mode = sys.argv[1] if len(sys.argv) > 1 else 'multiple'
149
+if mode == 'one':
150
+ app.layout = html.Div([
151
+ html.H1('System Monitoring Dashboard (Combined Graph)'),
152
+ # Combined Line Chart for RAM, CPU, and Disk
153
+ dcc.Graph(id='combined-graph'),
154
+ # Interval for updating the dashboard every 5 seconds
155
+ dcc.Interval(
156
+ id='interval-component',
157
+ interval=5*1000, # 5000 milliseconds (5 seconds)
158
+ n_intervals=0
159
+ )
160
+ ])
161
+ # Update callback to refresh the combined RAM, CPU, and Disk usage graph every interval
162
+ @app.callback(
163
+ Output('combined-graph', 'figure'),
164
+ [Input('interval-component', 'n_intervals')]
165
+ )
166
+ def update_combined_graph(n):
167
+ # Fetch system stats (RAM, CPU, and Disk)
168
+ data = get_system_stats()
169
+ if not data:
170
+ logging.info("No data fetched")
171
+ return {}
172
+ # Log fetched data in the terminal
173
+ logging.info(f"Fetched data: {data}")
174
+ # Append the current time, RAM, CPU, and Disk usage to history
175
+ current_time = datetime.now().strftime('%H:%M:%S') # Get the current time as a string
176
+ history['ram'].append(data['RAM Usage (%)'])
177
+ history['cpu'].append(data['CPU Usage (%)'])
178
+ history['disk'].append(data['Disk Usage (%)'])
179
+ history['time'].append(current_time)
180
+ # Create Combined Line Chart
181
+ combined_figure = {
182
+ 'data': [
183
+ go.Scatter(
184
+ x=list(history['time']),
185
+ y=list(history['ram']),
186
+ mode='lines+markers',
187
+ name='RAM Usage (%)'
188
+ ),
189
+ go.Scatter(
190
+ x=list(history['time']),
191
+ y=list(history['cpu']),
192
+ mode='lines+markers',
193
+ name='CPU Usage (%)'
194
+ ),
195
+ go.Scatter(
196
+ x=list(history['time']),
197
+ y=list(history['disk']),
198
+ mode='lines+markers',
199
+ name='Disk Usage (%)'
200
+ )
201
+ ],
202
+ 'layout': go.Layout(
203
+ title='RAM, CPU, and Disk Usage Over Time',
204
+ xaxis=dict(title='Time', tickformat='%H:%M:%S'), # Format the time
205
+ yaxis=dict(title='Percentage'),
206
+ )
207
+ }
208
+ return combined_figure
209
+else:
210
+ # Layout for multiple graphs (RAM, CPU, Disk each on its own graph)
211
+ app.layout = html.Div([
212
+ html.H1('System Monitoring Dashboard (Separate Graphs)'),
213
+ # RAM Usage Line Chart
214
+ dcc.Graph(id='ram-usage-graph'),
215
+ # CPU Usage Line Chart
216
+ dcc.Graph(id='cpu-usage-graph'),
217
+ # Disk Usage Line Chart
218
+ dcc.Graph(id='disk-usage-graph'),
219
+ # Interval for updating the dashboard every 5 seconds
220
+ dcc.Interval(
221
+ id='interval-component',
222
+ interval=5*1000, # 5000 milliseconds (5 seconds)
223
+ n_intervals=0
224
+ )
225
+ ])
226
+ # Update callback to refresh the RAM, CPU, and Disk usage graphs every interval
227
+ @app.callback(
228
+ [Output('ram-usage-graph', 'figure'),
229
+ Output('cpu-usage-graph', 'figure'),
230
+ Output('disk-usage-graph', 'figure')],
231
+ [Input('interval-component', 'n_intervals')]
232
+ )
233
+ def update_separate_graphs(n):
234
+ # Fetch system stats (RAM, CPU, and Disk)
235
+ data = get_system_stats()
236
+ if not data:
237
+ logging.info("No data fetched")
238
+ return {}, {}, {}
239
+ # Log fetched data in the terminal
240
+ logging.info(f"Fetched data: {data}")
241
+ # Append the current time, RAM, CPU, and Disk usage to history
242
+ current_time = datetime.now().strftime('%H:%M:%S') # Get the current time as a string
243
+ history['ram'].append(data['RAM Usage (%)'])
244
+ history['cpu'].append(data['CPU Usage (%)'])
245
+ history['disk'].append(data['Disk Usage (%)'])
246
+ history['time'].append(current_time)
247
+ # Create RAM Usage Line Chart
248
+ ram_figure = {
249
+ 'data': [go.Scatter(
250
+ x=list(history['time']),
251
+ y=list(history['ram']),
252
+ mode='lines+markers',
253
+ name='RAM Usage (%)'
254
+ )],
255
+ 'layout': go.Layout(
256
+ title='RAM Usage Over Time',
257
+ xaxis=dict(title='Time', tickformat='%H:%M:%S'), # Format the time
258
+ yaxis=dict(title='Percentage'),
259
+ )
260
+ }
261
+ # Create CPU Usage Line Chart
262
+ cpu_figure = {
263
+ 'data': [go.Scatter(
264
+ x=list(history['time']),
265
+ y=list(history['cpu']),
266
+ mode='lines+markers',
267
+ name='CPU Usage (%)'
268
+ )],
269
+ 'layout': go.Layout(
270
+ title='CPU Usage Over Time',
271
+ xaxis=dict(title='Time', tickformat='%H:%M:%S'), # Format the time
272
+ yaxis=dict(title='Percentage'),
273
+ )
274
+ }
275
+ # Create Disk Usage Line Chart
276
+ disk_figure = {
277
+ 'data': [go.Scatter(
278
+ x=list(history['time']),
279
+ y=list(history['disk']),
280
+ mode='lines+markers',
281
+ name='Disk Usage (%)'
282
+ )],
283
+ 'layout': go.Layout(
284
+ title='Disk Usage Over Time',
285
+ xaxis=dict(title='Time', tickformat='%H:%M:%S'), # Format the time
286
+ yaxis=dict(title='Percentage'),
287
+ )
288
+ }
289
+ return ram_figure, cpu_figure, disk_figure
290
+# Run the app
291
+if __name__ == '__main__':
292
+ app.run_server(debug=True)
293
+```
294
+
295
+
296
+Step 5: Running the Dashboard
297
+-----------------------------
298
+
299
+1. Ensure your virtual environment is activated.
300
+2. Run the dashboard with the following command to show one chart with all three metrics:
301
+
302
+```
303
+python app.py one
304
+
305
+```
306
+
307
+
308
+3\. Run the dashboard with the following command to show three charts each with one metric:
309
+
310
+```
311
+python app.py multiple
312
+```
313
+
314
+
315
+This will start the application on a local web server. Open your browser and visit:
316
+
317
+```
318
+http://127.0.0.1:8050/
319
+```
320
+
321
+
322
+You’ll see a dashboard that updates every 5 seconds, showing:
323
+
324
+* **RAM Usage** in percentage
325
+* **CPU Usage** in percentage
326
+* **Disk Usage** in percentage
327
+
328
+Step 6: Additional Notes
329
+------------------------
330
+
331
+* **psutil** works cross-platform, so the code will run on both Windows and Linux without modification.
332
+* If you’re on a macOS system, all functions except disk usage should work. You might need to adjust `psutil.disk_usage('/')` if you have a different filesystem configuration.
333
+
334
+Breakdown of Above Code:
335
+------------------------
336
+
337
+1\. Setting Up the Logging System
338
+---------------------------------
339
+
340
+The logging configuration is initialized early in the script, allowing us to capture and print important debugging information to the console.# Set up basic logging to debug
341
+
342
+```
343
+# Set up basic logging to debug
344
+logging.basicConfig(level=logging.INFO)
345
+```
346
+
347
+
348
+This line configures the logging system to show messages at the `INFO` level or higher, providing useful feedback during the program's execution.
349
+
350
+2\. Initializing the Dash Application
351
+-------------------------------------
352
+
353
+Dash is a powerful Python framework for building interactive web applications. The app is initialized as follows:
354
+
355
+```
356
+# Initialize the Dash app
357
+app = dash.Dash(__name__)
358
+```
359
+
360
+
361
+This creates a Dash application, which will serve as the basis for the real-time dashboard. The `__name__` parameter helps Dash locate resources correctly.
362
+
363
+3\. Setting Up History with Deques
364
+----------------------------------
365
+
366
+Deques (double-ended queues) are used to store the last 20 data points for RAM, CPU, Disk usage, and timestamps.
367
+
368
+```
369
+# Define fixed-size lists (deque) to store the last 20 data points for RAM, CPU, Disk usage, and time
370
+history = {
371
+ 'ram': deque(maxlen=20),
372
+ 'cpu': deque(maxlen=20),
373
+ 'disk': deque(maxlen=20),
374
+ 'time': deque(maxlen=20) # Store timestamps for x-axis
375
+}
376
+```
377
+
378
+
379
+The `maxlen=20` ensures that only the last 20 values are kept in memory, and older values are automatically removed. This is particularly useful for real-time graphs, which only need to show a limited number of recent data points.
380
+
381
+4\. Fetching System Statistics
382
+------------------------------
383
+
384
+We use the `psutil` library to gather system data such as RAM, CPU, and disk usage. The `get_system_stats` function is responsible for this:
385
+
386
+```
387
+def get_system_stats():
388
+ try:
389
+ # Get memory stats
390
+ memory = psutil.virtual_memory()
391
+ ram = memory.percent
392
+ # Get CPU usage
393
+ cpu = psutil.cpu_percent(interval=1)
394
+ # Get Disk usage
395
+ disk = psutil.disk_usage('/').percent
396
+ # Return RAM, CPU, and Disk data
397
+ return {
398
+ 'RAM Usage (%)': ram,
399
+ 'CPU Usage (%)': cpu,
400
+ 'Disk Usage (%)': disk
401
+ }
402
+ except Exception as e:
403
+ logging.error(f"Error fetching system stats: {e}")
404
+ return {}
405
+```
406
+
407
+
408
+**RAM Usage**: We call `psutil.virtual_memory()` to get memory information and extract the percentage of RAM in use.
409
+
410
+**CPU Usage**: The `psutil.cpu_percent()` function returns the current CPU usage. The `interval=1` argument tells the function to calculate CPU usage over a 1-second period.
411
+
412
+**Disk Usage**: We use `psutil.disk_usage('/')` to get the disk usage percentage for the root directory (`/`).
413
+
414
+This function gathers all this data and returns it as a dictionary. If an error occurs, it logs the error and returns an empty dictionary.
415
+
416
+5\. Mode Selection: Single or Multiple Graphs
417
+---------------------------------------------
418
+
419
+The dashboard can operate in two modes: either all data is combined in a single graph, or each metric (RAM, CPU, and Disk) has its own graph. This is determined by checking the command-line arguments:
420
+
421
+```
422
+# Determine whether to run in 'one' or 'multiple' mode based on command-line argument
423
+mode = sys.argv[1] if len(sys.argv) > 1 else 'multiple'
424
+```
425
+
426
+
427
+If the argument `one` is provided when the script is run, the application will combine all the data into a single graph. If no argument or `multiple` is provided, separate graphs will be displayed.
428
+
429
+6\. Creating the Layout and Callbacks for Combined Graph Mode
430
+-------------------------------------------------------------
431
+
432
+When running in the `one` mode, the layout contains a single graph, and the data is refreshed every 5 seconds:
433
+
434
+```
435
+if mode == 'one':
436
+ app.layout = html.Div([
437
+ html.H1('System Monitoring Dashboard (Combined Graph)'),
438
+ # Combined Line Chart for RAM, CPU, and Disk
439
+ dcc.Graph(id='combined-graph'),
440
+ # Interval for updating the dashboard every 5 seconds
441
+ dcc.Interval(
442
+ id='interval-component',
443
+ interval=5*1000, # 5000 milliseconds (5 seconds)
444
+ n_intervals=0
445
+ )
446
+ ])
447
+```
448
+
449
+
450
+This layout includes a title (`html.H1`) and a graph component (`dcc.Graph`). The `dcc.Interval` component is used to refresh the data every 5 seconds (`5000 milliseconds`).
451
+
452
+The callback function below updates the combined graph by fetching the latest system stats and adding them to the `history` deques:
453
+
454
+```
455
+@app.callback(
456
+ Output('combined-graph', 'figure'),
457
+ [Input('interval-component', 'n_intervals')]
458
+)
459
+def update_combined_graph(n):
460
+ # Fetch system stats (RAM, CPU, and Disk)
461
+ data = get_system_stats()
462
+ if not data:
463
+ logging.info("No data fetched")
464
+ return {}
465
+ # Log fetched data in the terminal
466
+ logging.info(f"Fetched data: {data}")
467
+ # Append the current time, RAM, CPU, and Disk usage to history
468
+ current_time = datetime.now().strftime('%H:%M:%S') # Get the current time as a string
469
+ history['ram'].append(data['RAM Usage (%)'])
470
+ history['cpu'].append(data['CPU Usage (%)'])
471
+ history['disk'].append(data['Disk Usage (%)'])
472
+ history['time'].append(current_time)
473
+ # Create Combined Line Chart
474
+ combined_figure = {
475
+ 'data': [
476
+ go.Scatter(
477
+ x=list(history['time']),
478
+ y=list(history['ram']),
479
+ mode='lines+markers',
480
+ name='RAM Usage (%)'
481
+ ),
482
+ go.Scatter(
483
+ x=list(history['time']),
484
+ y=list(history['cpu']),
485
+ mode='lines+markers',
486
+ name='CPU Usage (%)'
487
+ ),
488
+ go.Scatter(
489
+ x=list(history['time']),
490
+ y=list(history['disk']),
491
+ mode='lines+markers',
492
+ name='Disk Usage (%)'
493
+ )
494
+ ],
495
+ 'layout': go.Layout(
496
+ title='RAM, CPU, and Disk Usage Over Time',
497
+ xaxis=dict(title='Time', tickformat='%H:%M:%S'), # Format the time
498
+ yaxis=dict(title='Percentage'),
499
+ )
500
+ }
501
+ return combined_figure
502
+```
503
+
504
+
505
+This callback fetches the data every 5 seconds, appends it to the `history` deque, and then creates a combined line chart to display the three metrics over time.
506
+
507
+7\. Creating the Layout and Callbacks for Multiple Graphs Mode
508
+--------------------------------------------------------------
509
+
510
+In `multiple` mode, the layout consists of three separate graphs (one for each metric):
511
+
512
+```
513
+else:
514
+ app.layout = html.Div([
515
+ html.H1('System Monitoring Dashboard (Separate Graphs)'),
516
+ # RAM Usage Line Chart
517
+ dcc.Graph(id='ram-usage-graph'),
518
+ # CPU Usage Line Chart
519
+ dcc.Graph(id='cpu-usage-graph'),
520
+ # Disk Usage Line Chart
521
+ dcc.Graph(id='disk-usage-graph'),
522
+ # Interval for updating the dashboard every 5 seconds
523
+ dcc.Interval(
524
+ id='interval-component',
525
+ interval=5*1000, # 5000 milliseconds (5 seconds)
526
+ n_intervals=0
527
+ )
528
+ ])
529
+```
530
+
531
+
532
+The callback in this mode updates each graph individually:
533
+
534
+```
535
+@app.callback(
536
+ [Output('ram-usage-graph', 'figure'),
537
+ Output('cpu-usage-graph', 'figure'),
538
+ Output('disk-usage-graph', 'figure')],
539
+ [Input('interval-component', 'n_intervals')]
540
+)
541
+def update_separate_graphs(n):
542
+ # Fetch system stats (RAM, CPU, and Disk)
543
+ data = get_system_stats()
544
+ if not data:
545
+ logging.info("No data fetched")
546
+ return {}, {}, {}
547
+ # Append the current time, RAM, CPU, and Disk usage to history
548
+ current_time = datetime.now().strftime('%H:%M:%S')
549
+ history['ram'].append(data['RAM Usage (%)'])
550
+ history['cpu'].append(data['CPU Usage (%)'])
551
+ history['disk'].append(data['Disk Usage (%)'])
552
+ history['time'].append(current_time)
553
+ # Create RAM, CPU, and Disk Usage Line Charts
554
+ ram_figure = {
555
+ 'data': [go.Scatter(
556
+ x=list(history['time']),
557
+ y=list(history['ram']),
558
+ mode='lines+markers',
559
+ name='RAM Usage (%)'
560
+ )],
561
+ 'layout': go.Layout(title='RAM Usage Over Time', xaxis=dict(title='Time'), yaxis=dict(title='Percentage'))
562
+ }
563
+ cpu_figure = {
564
+ 'data': [go.Scatter(
565
+ x=list(history['time']),
566
+ y=list(history['cpu']),
567
+ mode='lines+markers',
568
+ name='CPU Usage (%)'
569
+ )],
570
+ 'layout': go.Layout(title='CPU Usage Over Time', xaxis=dict(title='Time'), yaxis=dict(title='Percentage'))
571
+ }
572
+ disk_figure = {
573
+ 'data': [go.Scatter(
574
+ x=list(history['time']),
575
+ y=list(history['disk']),
576
+ mode='lines+markers',
577
+ name='Disk Usage (%)'
578
+ )],
579
+ 'layout': go.Layout(title='Disk Usage Over Time', xaxis=dict(title='Time'), yaxis=dict(title='Percentage'))
580
+ }
581
+ return ram_figure, cpu_figure, disk_figure
582
+```
583
+
584
+
585
+Each graph displays its own data set, pulled from the `history` deque and updated every 5 seconds.
586
+
587
+8\. Running the App
588
+-------------------
589
+
590
+Finally, the Dash app is started using the `app.run_server()` function:
591
+
592
+```
593
+if __name__ == '__main__':
594
+ app.run_server(debug=True)
595
+```
596
+
597
+
598
+Running the app with `python app.py one` will display a combined graph, while `python app.py multiple` will display separate graphs for each metric.
599
+
600
+Visit this [link](https://medium.com/p/35613932d123) for a similar project that uses Chart.js rather than dash/plotly.
601
+
602
+Thank you for reading this article. I hope you found it helpful and informative. If you have any questions, or if you would like to suggest new Python code examples or topics for future tutorials, please feel free to reach out. Your feedback and suggestions are always welcome!
603
+
604
+Happy coding!
605
+C. C. Python Programming
... ...
\ No newline at end of file