Top 10 Container(Docker) Monitoring Solutions and Tools in 2018

Top 10 Container Monitoring Solutions/Tools in 2018

  1. Native Docker
  2. cAdvisor
  3. Scout
  4. Pingdom
  5. Datadog
  6. Sysdig
  7. Prometheus
  8. Heapster / Grafana
  9. ELK stack
  10. Sensu

Reference

https://rancher.com/comparing-10-container-monitoring-solutions-rancher/

Tagged : / / / /

Top 10 Infrastructure Monitoring Tools | List of Best Infrastructure Monitoring Tools

infrastructure-monitoring-tools
There are various types of tools used by DevOps professionals in their work environment from Continuous integration tools to Virtualization to configuration Management to monitoring and the list goes on and on. So today, we are going to talk about Infrastructure Monitoring tools. Which plays a vital role in the DevOps culture.
 
First of all lets see what is Infrastructure Monitoring ?
 
Infrastructure monitoring is a process in which Organisations IT framework is monitor by companies with the help of tools and softwares. This is essential so that you can make sure that your product availability is good and providing efficient services to customers or users. In other words, Infrastructure Monitoring tools help organisations to find and short-out infrastructure issues before it affect their businesses. It provides the insight status of physical, virtual, and cloud systems which helps in management of these things which ultimately helps in business processes.
 
Now, Let’s check out the major benefits of Infrastructure Monitoring Tools ?
 
1. Infrastructure monitoring tools help to detect the technical problems or outages before it occurs which helps engineers to act in advance accordingly so that it can not affect other things.
 
2. It also helps in quick troubleshooting. When problems happen the first thing engineers do is to troubleshoot them but they cannot do it without proper reports of the infrastructures and monitoring provides them end-to-end informations which helps them to short out the problem quickly.
 
3. This is beneficial when you need to go back to the history to find and choose amongst the infrastructure options for your organization. You can check all the previous monitoring details which helps to take better and firm decisions while selecting.
 
4. It helps in productivity of the human resources by providing them the automation which ultimately helps them to focus on the other things of the infrastructure management and making other strategies.
 
5. All these things which mentioned above ultimately helps the end users to use services smoothly which is very much necessary for running your business effectively and efficiently.
 
 
Now, as we all know there are various tools you can find for Infrastructure monitoring but to choose amongst them you need to do some research. But, don’t waste your time on research as I have done already the same and make my list of top 5 infrastructure monitoring tools.
 
Here is the list:-
 

1. Anturis

Anturis

 

Anturis is a cloud based monitoring tool which belongs to Anturis Inc. It’s like an all-in-one infrastructure monitoring tools which provides server monitoring, network monitoring, website monitoring and application monitoring.
 
Highlights
  • On-premise and cloud-based systems monitoring
  • Seamless agent-based and agent-less monitoring
  • Single dashboard monitoring for multiple data centers in different locations
  • Add many infrastructures and components easily as per needs and can streamline for different users
  • Smart alerts and notification that helps to eliminate “false” alerts and alert “spam”
 

2. AppDynamics

AppDynamics

 
AppDynamics is basically an Application infrastructure monitoring tool which belongs to AppDynamics Inc. but now it’s acquired by Cisco
 
Highlight
  • Provides Visibility into Server Performance
  • Visibility into Database Performance
  • Low overhead monitoring for both
  • Available for both on-premise and SaaS deployments
  • License flexibility to migrate across deployment models

3. BigPanda

BigPanda

BigPanda is also an Infrastructure monitoring tool from bigpanda.io which is founded in 2012 by Assaf Resnick & Elik Eizenberg. It uses algorithms to consolidate data across multiple monitoring systems.
 
Highlights
 
  • Ops-Aware Inbox: All your alerts organized, & updated in one place
  • Reduce Noisy Alerts: Snooze non-actionable alerts
  • Auto-correlate related alerts into high level incidents
  • Collaboration: Share, assign and track alerts
  • Advanced reporting and analytics
  • Trends, real-time activity log, and historical data in just a few clicks
  • Customized views for any app, team or business service
 
4. Boundary

Boundary

Boundary was founded by Cliff Moon & Benjamin Blacka. It is a consolidated operations management platform providing SaaS-based solutions for clients running apps and cloud infrastructures.
 
Highlights
  • Real-time trends of critical metrics
  • Pre-built library of event connectors, APIs for others
  • Early Warnings
  • Dashboard View
  • Analytics for trend analysis and anomaly detection
  • Easy to visualize response times between application tiers
 
5. CopperEgg or Uptime Infrastructure Monitor

CopperEgg

Copper egg was founded Scott Johnson, Eric Anderson & Bob Quillin in 2010 which is later in 2013 acquired by IDERA. It is now renamed to Uptime Infrastructure Monitor. You can monitor physical servers, virtual machines, network devices, applications, and services across multiple platforms running on-premise, remotely, or in the Cloud with the help of Uptime Infrastructure Monitor.
 
Highlights
  • Unified IT Dashboard
  • Customizable, drag-and-drop dashboards
  • Reports on app & server uptime
  • Proactive alerts & root-cause analysis
  • Team or role-specific dashboards
  • Cloud, on-premise or remote monitoring
  • Cloud monitoring incl. Amazon’s EC2 & Rackspace
  • Agentless and agent-based server monitoring
 
 
6. Datadog

Datadog

Datadog was founded in the 2010 by Olivier Pomel & Alexis Lê-Quôc. It provides monitoring for cloud-scale applications. It brings the data together from applications, cloud providers and specialized management tools in one location.
 
Highlights
  • Free Trial for an unlimited number of hosts
  • Out-of-the-box and customizable monitoring dashboards
  • Alert notifications via e-mail and PagerDuty
  • Full API access in more than 15 languages
  • Tools for team collaboration
  • Easy-to-use search for hosts, metrics, and tags
 
 
7. Nagios

Nagios

Nagios is a veteran Infrastructure monitoring tool which was released in 1999 which provides monitoring and alerting solutions for servers, switches, applications and services.
 
Highlights
  • Open source tool
  • Comprehensive monitoring
  • Powerful script APIs
  • Performance & capacity planning graphs
  • Advanced graphs & visualizations
  • Advanced user management
  • Extendable architecture
  • Reporting
 
8. New Relic

New Relic

New relic provides Complete dynamic infrastructure and server monitoring which is founded by Lew Cirne in 2008.
 
Highlights
  • Real-time health metrics for all your systems
  • Live-state Event Feed and a complete change history across all your hosts
  • Slice-and-dice by AWS tags, custom attributes, and metadata
  • Tag-driven alerting and dashboarding for all your dynamic resources
  • Infrastructure-wide search to find vulnerable packages or other resources
  • Docker support, including the ability to track container performance by image, version, and other labels
 
9. Icinga

Icinga

Icinga is an open source monitoring tool which was released in 2009 was originally created as a fork of the Nagios system monitoring application.
 
Highlights
  • Wealth of community-developed plugins
  • Stable codebase.
  • Monitoring of Network,Host and Server Components
  • Notification of contact persons when service or host problems occur and get resolved
  • Alerts transfer to other users or channels
  • Template based reports
  • Report repository with varying access levels and automated report generation and distribution
  • Clear-cut, object-based configuration
  • Clever commands & runtime macros
  • Apply & assign attributes
 
10. AppNeta

AppNeta

AppNeta also provides monitoring services and this was founded in 2000 by Irfhan Rajani and Matt Stevens. It’s provide SaaS-based application and network monitoring for any cloud, for every user and for all locations.
 
Highlights
  • Detailed performance trends in real-time
  • Real-User Monitoring
  • synthetic trends
  • Troubleshoot issues across the entire app network path
  • Proactive Alerts
  • Network Performance Monitoring
  • Provides visibility into AWS components, as well as the network they communicate over
 
So, That’s it. This is my list of top infrastructure monitoring tools and it will definitely help you while choosing them. But, if you think this list should contain any other tools instead of this than please share your list or views in comment section below.

Reference

Tagged : / / / / / / / / / / / / / / / / / / /

Top 5 Log Monitoring Tools | List of Log Monitoring Tools | scmGalaxy

These days there are various kinds of tools used by professionals in Software industry. From CI tools to Virtualization, to Issues, bug tracking tools to various others tools, the lists goes on and on. Thus, it’s really a tough task for them to select best tools as per their needs. So, today I am going to help System Administrators and operations by providing them the lists of top log monitoring tools.
But before that lets see, what is log monitoring ?
Log monitoring is a process of monitoring and understanding the state of the system, server, network, security devices and different applications and how they are working. Actually when these servers and different applications runs, they generate log files and events like Errors, problems, and more information is constantly logged and saved for analysis as text or binary files in the system. System admin keep reviewing these log files on daily basis so they can understand the condition and functioning of systems and different applications. But reviewing all those log files on daily basis is not an easy task, to do this laborious task log monitor needs to be set on those logs which automatically
monitor, review, analyze and generate reports as per the instructions set by System Admin.
As you saw how helpful this is for system admin but to find a good tool for this process is not an easy task. So, without wasting any time let’s check out the top 5 monitoring tools.
1. Logstash
logstash
Features
  • Extended functionality via plugins.
  • Free and open source.
  • Filters are code.
  • Great integration with other Elastic products.

2. Fluentd
fluentd
Features
  • Handles up to 50,000 messages per second at peak time
  • Data filtering and alerting
  • Apache 2.0 License project
  • Simplify and scale data pipeline management with tags
  • More than 200 community-contributed plugins
  • Real-time monitoring and alerts system
  • Store data in multiple systems
  • Collect and correlate web server access logs and application error logs

 

3. Loggly

loggly
Features
  • Text-based logs from any source
  • Custom source groups
  • Point-and-click trending graphs
  • Automated filters and event parsing
  • Full-system RESTful API to integrate with other applications
  • Unlimited saved searches and users
  • Adaptable interface with multiple views, pages and workspaces
  • Unlimited custom dashboards based on any search
  • Built-in customizable alerts with triggers

4. Graylog2
graylog2
Features
  • Leverages Java, Scala and ElasticSearch technologies
  • Central syslog monitoring
  • Interactive API browser
  • Application debugging
  • Exception monitoring
  • API analytics
  • Intuitive search interface
  • Comprehensive dashboard

5. Splunk
splunk
Features
  • Works with Hadoop & NoSQL
  • Setup standard searches as real-time alerts
  • Trigger automatic responses
  • Scale from a single server to multiple data centers
  • Deployable to on premise sites, hybrid-cloud, and/or private and public cloud based infrastructures
  • Securely make data available
  • Supports various use cases including log consolidation/retention, security, compliance reporting, and more
  • Perform ad hoc queries
So, this is the list my top log monitoring tools, I hope you find this useful and if not than feel free to share your thoughts in comment section below.
Tagged : / / / / / / / / / / / / / / /

Server configuration monitoring and diagnostics

server-configuration-monitoring-and-diagnostics

ScriptRock’s GuardRail

As DevOps gets more agile, infrastructure needs unit tests too. Enterprise startup ScriptRock offers cloud-hosted testing configuration tools to get you started.

Tagged : / / / / / /

Linux Performance Monitoring using iostat, mpstat and vmstat | Linux Performance Monitoring Guide

linux-performance-monitoring

iostat reports CPU, disk I/O, and NFS statistics.

mpstat reports processors statictics.
vmstat reports virtual memory statistics.

vxstat – This Utility can be used as well.

iostat reports CPU, disk I/O, and NFS statistics.

The iostat command generates two types of reports, the CPU Utilization report and the Device Utilization report. The first report generated by the iostat command is the CPU Utilization Report.

iostat commands Table
iostat Iostat without any argument displays information about the CPU usage, and I/O statistics
iostat -c Display only cpu statistics
iostat -d Display only disk I/O statistics
iostat -n Display only network statistics (device and NFS statistics)
iostat -m Display the device I/O statistics in Blocks and To change I/O data in MB/second
iostat –p Display I/O statistics only for a device(data for all the disks available in the system)
iostat -t By default iostat displays only the current date. To display the current time, use this option
iostat -x Display extended disk I/O statistics information
iostat -x sda1 Display Extended status of device sda1
iostat 2 Execute Every 2 seconds (until you press Ctl-C)
iostat 2 3 To execute every 2 seconds for a total of 3 times
iostat -N To display the LVM statistics use option -N as shown below.
iostat –V To Display Version
iostat -d -x Display only disk I/O statisticsin in extended form
iostat -d -x sda Display only sda device I/O statisticsin in extended form
iostat -Td -xdn 5 For Solaris
iostat -d -x 5 For Linux
if Average read or write times over 50 ms, this should be consider for serious i/o problems

The report has the following format:

%user – Show the percentage of CPU utilization that occurred while executing at the user level (application).
%nice – provides statistics on a per physical device or partition basis.
Device – This column gives the device name
tps – Indicate the number of transfers per second that were issued to the device. A transfer is an I/O request to the device.
Blk_read/s – Indicate the amount of data read from the drive expressed in a number of blocks per second.
Blk_wrtn/s – Indicate the amount of data written to the drive expressed in a number of blocks per second.
Blk_read – The total number of blocks read.
Blk_wrtn – The total number of blocks written.
kB_read/s – Indicate the amount of data read from the drive expressed in kilobytes per second.
kB_wrtn/s – Indicate the amount of data written to the drive expressed in kilobytes per second.
wrqm/s – The number of write requests merged per second that were issued to the device.
r/s – The number of read requests that were issued to the device per second.
w/s – The number of write requests that were issued to the device per second.
rsec/s – The number of sectors read from the device per second.
wsec/s – The number of sectors written to the device per second.
await – The average time (in milliseconds) for I/O requests issued to the device to be served.
svctm – The average service time (in milliseconds) for I/O requests that were issued to the device.
%util – Percentage of CPU time during which I/O requests were issued to the device.

vmstat reports virtual memory statistics.

vmstat -a Display active and inactive memory
vmstat -f Display number of forks since last boot
vmstat 2 Execute Every x seconds (for y number of times)
vmstat -t 1 100 Display timestamp
vmstat -V version info
vmstat -m Display slab info
vmstat -s Display statistics in a table format
vmstat -d Use option -d to display the disk statistics as shown below. This displays the reads, writes, and I/O statistics of the disk.
vmstat 1 3 vmstat – Increase the width of the display
vmstat -p sdb1 Display statistics for a partition
vmstat -S m Display in MB

The report has the following format:

Procs – r: Total number of processes waiting to run
Procs – b: Total number of busy processes
Memory – swpd: Used virtual memory
Memory – free: Free virtual memory
Memory – buff: Memory used as buffers
Memory – cache: Memory used as cache.
Swap – si: Memory swapped from disk (for every second)
Swap – so: Memory swapped to disk (for every second)
IO – bi: Blocks in. i.e blocks received from device (for every second)
IO – bo: Blocks out. i.e blocks sent to the device (for every second)
System – in: Interrupts per second
System – cs: Context switches
CPU – us, sy, id, wa, st: CPU user time, system time, idle time, wait time

mpstat reports processors statictics.

reports processors statictics(mpstat – Report processors related statistics and mpstat displays CPU statistics )

mpstat :- mpstat displays CPU statistics
mpstat -A :- Display all information
mpstat -P ALL :- Display CPU statistics of individual CPU (or) Core

Reference:
http://www.linux-tutorial.info/modules.php?name=ManPage&sec=1&manpage=iostat
http://www.thegeekstuff.com/2011/07/iostat-vmstat-mpstat-examples/

Tagged : / / / / / / / / / / / / / / / / / / /

The symptoms of our software development malaise

software-development-malaise

Software development has traditionally suffered from producing end products with a definite lack of inherent quality. The symptoms of this quality lack are listed here:

  • Software development projects are often delivered late and over budget.
  • Often the delivered product does not meet customer requirements and is never used.
  • Software products simply do not work right.

As we look into the symptoms of our software development malaise, five principal issues related to software development arise.

Lack of Visibility
Software is conceptual in nature. Unlike a bridge, a building, or another physical structure, it is not easy to look at software and assess how close it is to completion. Without strong project management, “software is 90% complete 90% of the time.” Through the adoption of SCM policy and the definition of the configuration management model of the software under development, all CIs, components, and subcomponents are immediately visible for versions, releases, and product families.


Lack of Control
Because software is inherently intangible, it is also more difficult to control. Without an accurate assessment of progress, schedules slip and budgets are overrun. It is hard to assess what has been accomplished and what remains to be done. SCM provides the mechanism for controlling the project through measuring the amount of effort compared to the project management plan and estimating the future effort based on past work.

Lack of Traceability
A lack of linkage between project events contributes to project failures. The main benefit of SCM is providing the traceability among versions, releases, and product families. The value of this traceability is enormous when a problem arises in one release or product family that impacts other client releases and products. Making one change and promoting that through the entire product software base is an incredible cost savings in time, money, and client good will. A lack of linkage between project events contributes to project failures where solving one problem either exacerbates a problem in another area or fails to fix the same problem elsewhere. A traceability thread allows management to examine the chain of events that caused a project to encounter difficulty as an integral process within the auditing capability of SCM. A project becomes a year late one day at a time unless the effort reported on the schedule maps to the actual work being done and traced within the software configuration management system.

Lack of Monitoring
Without traceability and visibility, monitoring of software development projects becomes extremely difficult. Management cannot make informed decisions, and thus schedules slip further and costs continue to exceed budget.
There is no way to monitor a project when the project manager has no tools to look into the actual product development within the project. SCM provides the tools that open up the process to external monitoring. With SCM in place and a policy of traceability and visibility accepted, monitoring of software development projects becomes a simple part of the overall project management task. Management makes informed decisions avoiding schedule slips and budget excesses through the monitoring available with SCM tools and the integral workings of the CCB.

Uncontrolled Change
Software is very malleable; it is idea-stuff, and customers constantly have new ideas for it. People would rarely ask a bridge constructor to make the kinds of changes midproject that software customers tend to request. The impact of such changes can be just as great. All SCM tools, along with the CCB, support a mechanism for appropriate change control.

SCM Interacts with Verification and Validation
SCM is most important, and most often neglected, during V&V activities, which include software testing. It is employed in tracking which module versions are included in a particular system build, as well as which tests have been run. The results of the tests are tied directly to the modules or subcomponents being tested. Many times there are “action items” resulting from the tests. SCM tracks the action item status, so overall system status can be assessed well before final builds and releases are done.
Verification and validation testing are supported through the four basic SCM processes: identification, control, auditing, and status accounting. Let’s look at examples of V&V testing in the context of each of these components.

SCM Identification Benefits to V&V

  • Automatic preparation of release notes
  • List of changed software modules
  • Definition of development baseline
  • Generation of incident reports
  • Definition of operational baseline
  • Control of the configuration item identification
  • Management of CCB meetings
  • Prioritization of test and incident issues
  • Establishment of turnover dates
  • Approval of audit and test reports
  • Approval of incident report resolutions

SCM Auditing Benefits to V&V

  • Comparison of new baseline to previous baseline
  • Assurance that standards have been met
  • Audit trail of the testing process (verification, validation, and acceptance) of the software system
  • Documentation of experience with technical aspects of system engineering or software engineering

SCM Status Accounting Benefits to V&V

  • Logging and tracking of incident reports
  • Publication of CCB minutes

With all of these potential benefits of SCM, project managers must address real-world considerations. Management commitment is the real key to implementing SCM on a specific project in a given organization. By treating the implementation of SCM as a project, the project plan must start at the top to secure commitment to checks and balances. Now is the time to bring out the organization’s list of project disasters to draw on management experience with their previous software project difficulties. If there are no historic disasters in the organization, or if it is inappropriate to discuss them, refer to literature articles that provide accounts of project disasters (refer to the Web resources at the end of this chapter). Finally, after putting a notional return-on-investment financial argument in place, explain the intangible benefits of SCM.
One of the major sources intangible benefits is auditing. Auditing can be a heavy consumer of configuration management resources, and management may question the benefit of this kind of expenditure. Auditing pays for itself through the avoidance of larger, unnecessary expenses. Savings of 100:1 (large projects) or 4–6:1 (small projects) are obtained by finding and fixing problems early in the life cycle. The auditing process in SCM uncovers the areas where a little more effort or control will result in higher-quality software products and lower overall project costs.
There can be audit compromises to reduce costs. As a project manager, plan audits based on both the phases of the life cycle and the frequency of builds, versions, releases, and product families. Auditing each baseline in a project while reducing the depth of each audit maintains some traceability with loss of visibility.
Eliminating one or more audits (installation baseline, for example) maintains visibility but slightly impacts traceability.

Tagged : / / / / / / / / / / / / / / / /