Could not retrieve mirrorlist http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=os&infra=stock error was
14: curl#6 - "Could not resolve host: mirrorlist.centos.org; Name or service not known"

On July 1, 2024, CentOS 7 reached end of life, and the CentOS team has moved its repositories to the archive at vault.centos.org. Without updating the repository URLs, packages cannot be updated or validated, resulting in these errors.

Solution, run script:

/scripts/centos7_fix_repository

First stop the future updates via yum/dnf package manager, if you skip this step then after each update of curl from the base repo you need to rebuild curl

Centos 7 /EL7 :

cat /etc/yum.conf |grep "^exclude="|grep kernel 1> /dev/null 2> /dev/null || echo 'exclude=curl* libcurl*' >> /etc/yum.conf

Centos 8 stream /EL8 :

cat /etc/dnf.conf |grep "^exclude="|grep kernel 1> /dev/null 2> /dev/null || echo 'exclude=curl* libcurl*' >> /etc/dnf.conf

Now you need to install dependencies to build curl and libcurl:

Centos 7 /EL7 :

yum install libssh libssh-devel libnghttp2-devel libnghttp2 libgsasl libgsasl-devel zstd libzstd-devel libzstd brotli brotli-devel libbrotli 

Centos 8 stream /EL8 :

dnf install libssh libssh-devel libnghttp2-devel libnghttp2 libgsasl libgsasl-devel zstd libzstd-devel libzstd brotli brotli-devel libbrotli 

After you've installed the dependencies build CURL from the source :

cd /usr/local/src
rm -rf curl*
wget https://curl.se/download/curl-7.88.0.zip
unzip curl-7.88.0.zip
cd curl-7.88.0
./configure --with-ssl --with-zlib --with-gssapi --enable-ldap --enable-ldaps --with-libssh --with-nghttp2
make
make install

After the successful build check the cURL version :

curl -V

List partitions and usage

df -h

Check Disk Usage per User

repquota -a -s

Show only accounts with usage in Gbytes

repquota -a -s | grep G

Check Disk Usage per Partition ignoring links
* You can replace /home with the parition or folder you want to check
Let's check /home partition

du -hx --max-depth 1 /home

Show only folders with usage in Gbytes

du -hx --max-depth 1 /home | grep G

Check /var partition

du -hx --max-depth 1 /var

Show only folders with usage in Gbytes

du -hx --max-depth 1 /var | grep G

Check Mail folder /var/vmail

du -hx --max-depth 1 /var/vmail

Show only folders with usage in Gbytes

du -hx --max-depth 1 /var/vmail | grep G

Check MySQL folder /var/lib/mysql

du -hx --max-depth 1 /var/lib/mysql

Show only folders with usage in Gbytes

du -hx --max-depth 1 /var/lib/mysql | grep G

Check Backups

du -hx --max-depth 1 /backup

Show only folders with usage in Gbytes

du -hx --max-depth 1 /backup | grep G

Show only daily backup folder with usage in Gbytes

du -hx --max-depth 1 /backup/daily | grep G

Now when you know how to check the partitions and folders here are some useful commands to check files in the folder sorted by size:

ls -lahS /folder/location

Example for MySQL folder /var/lib/mysql/

ls -lahS /var/lib/mysql/

Some useful scripts we already have are checking the most frequent folders:
/var/log/
/usr/local/apache/logs/
/usr/local/apache/domlogs/
/usr/local/cwpsrv/logs/
/tmp
/root
/var/lib/mysql/

/scripts/disk_check

Delete Files
Now when you know how to check, the question is how can you delete or empty files.
Note that you shouldn't delete files other than logs or backups as all others can crash your server.

Log files are in use by the programs so in case you want to clean big active log files you can't just delete them as they will remain invisible until you restart the service.
The solution is to empty the file, example:

:> /usr/local/apache/logs/access_log

However all other logs files you can simply delete, examples

rm -f  filename

An example case when you have active log files and logrotated which are not active and are renamed with -DATE

Delete single log file NOT currently in use by service

rm -f /var/log/maillog-20211108

Delete all maillog files NOT currently in use by service

rm -f /var/log/maillog-*

Empty active log file which is in use by the service

:> /var/log/maillog

* If you are not sure it's better to grab a support service and have the experienced sysadmin to check this for you.

If you are getting the account suspended message it can be for the following reasons:
- Account was suspended by server admin or automated billing system.
- Account HTTP/HTTPS is only suspended, this means the account monthly bandwidth limit was exceeded.

Account Suspended fully
If the account shows as suspended in the cwp.admin under the account list then you can unsuspend it also from there.
Mostly the reason for the suspension is Manual by SystemAdmin or by an Automated Billing system.

Account HTTP/HTTPS Suspended only
If only website access was suspended meaning that you can login to user panel, ftp, and email then HTTP/HTTPS for the account was suspended because the Bandwidth limit was exceeded.

You can UnSuspend bandwidth exceeded account by increasing the bandwidth limit of the package that account uses.
This change needs to be done from cwp.admin -> Packages module.

There is also an API that can unsuspend accounts but this will be temporary until the next bandwidth check cron which runs every 24h.

API functions:

UnSuspend Account:

/scripts/cwp_api account unsuspend_user USERNAME

UnSuspend Account Bandwidth (You should increase user/package bandwidth before):

/scripts/cwp_api account unsuspend_bandwidth USERNAME

Reset Account Bandwidth (It will reset user bandwidth to 0 and UnSuspend Account):

/scripts/cwp_api account reset_bandwidth USERNAME

If the suspended message continues then try to check vhost templates.

Elasticsearch is a platform for distributed search and analysis of data in real-time. Its popularity is due to its ease of use, powerful features, and scalability.

Elasticsearch supports RESTful operations. This means that you can use HTTP methods (GET, POST, PUT, DELETE, etc.) in combination with an HTTP URI (/collection/entry) to manipulate your data. The intuitive RESTful approach is both developer and user-friendly, which is one of the reasons for Elasticsearch’s popularity.

Elasticsearch is free and open-source software with a solid company behind it — Elastic. This combination makes it suitable for use anywhere from personal testing to corporate integration.

This article will introduce you to Elasticsearch and show you how to install, configure, and start using it.

Prerequisites

Before following this tutorial, please make sure you complete the following prerequisites:

A CentOS 7 Droplet
A non-root sudo user (Check out Initial Server Setup with CentOS 7 for details.)

Except otherwise noted, all of the commands that require root privileges in this tutorial should be run as a non-root user with sudo privileges.

Assumptions

This tutorial assumes that your servers are using a VPN like the one described here: How To Use Ansible and Tinc VPN to Secure Your Server Infrastructure. This will provide private network functionality regardless of the physical network that your servers are using.

If you are using a shared private network, such as DigitalOcean Private Networking, you must use a VPN to protect Elasticsearch from unauthorized access. Each server must be on the same private network because Elasticsearch doesn’t have security built into its HTTP interface. The private network must not be shared with any computers you don’t trust.

Step 1 — Installing Java

First, you will need a Java Runtime Environment (JRE) on your Droplet because Elasticsearch is written in the Java programming language. You can use the native CentOS OpenJDK package for the JRE. This JRE is free, well-supported, and automatically managed through the CentOS Yum installation manager.

You can install the latest OpenJDK with the command:

sudo yum install java-1.8.0-openjdk.x86_64

To verify your JRE is installed and can be used, run the command:

java -version

The result should look like this:

Output of java -version
openjdk version "1.8.0_65"
OpenJDK Runtime Environment (build 1.8.0_65-b17)
OpenJDK 64-Bit Server VM (build 25.65-b01, mixed mode)

When you advance in using Elasticsearch and you start looking for better Java performance and compatibility, you may opt to install Oracle’s proprietary Java (Oracle JDK 8). For more information on how to do this, please check the article on How To Install Java on CentOS and Fedora.

Step 2 — Downloading and Installing Elasticsearch

Elasticsearch can be downloaded directly from elastic.co in zip, tar.gz, deb, or rpm packages. For CentOS, it’s best to use the native rpm package which will install everything you need to run Elasticsearch.

At the time of this writing, the latest Elasticsearch version is 1.7.3. Download it in a directory of your choosing with the command:

wget https://download.elastic.co/elasticsearch/elasticsearch/elasticsearch-1.7.3.noarch.rpm

Then install it in the usual CentOS way with the rpm command like this:

sudo rpm -ivh elasticsearch-1.7.3.noarch.rpm

This results in Elasticsearch being installed in /usr/share/elasticsearch/ with its configuration files placed in /etc/elasticsearch and its init script added in /etc/init.d/elasticsearch.

To make sure Elasticsearch starts and stops automatically with the Droplet, add its init script to the default runlevels with the command:

sudo systemctl enable elasticsearch.service

Step 3 — Configuring Elastic

Now that Elasticsearch and its Java dependencies have been installed, it is time to configure Elasticsearch.

The Elasticsearch configuration files are in the /etc/elasticsearch directory. There are two files:

elasticsearch.yml — Configures the Elasticsearch server settings. This is where all options, except those for logging, are stored, which is why we are mostly interested in this file.
logging.yml — Provides configuration for logging. In the beginning, you don’t have to edit this file. You can leave all default logging options. You can find the resulting logs in /var/log/elasticsearch by default.

The first variables to customize on any Elasticsearch server are node.name and cluster.name in elasticsearch.yml. As their names suggest, node.name specifies the name of the server (node) and the cluster to which the latter is associated.

If you don’t customize these variables, a node.name will be assigned automatically in respect to the Droplet hostname. The cluster.name will be automatically set to the name of the default cluster.

The cluster.name value is used by the auto-discovery feature of Elasticsearch to automatically discover and associate Elasticsearch nodes to a cluster. Thus, if you don’t change the default value, you might have unwanted nodes, found on the same network, in your cluster.

To start editing the main elasticsearch.yml configuration file:

sudo nano /etc/elasticsearch/elasticsearch.yml

Remove the # character at the beginning of the lines for node.name and cluster.name to uncomment them, and then change their values. Your first configuration changes in the /etc/elasticsearch/elasticsearch.yml file should look like this:

/etc/elasticsearch/elasticsearch.yml

...
node.name: "My First Node"
cluster.name: mycluster1
...

Another important setting is the role of the server, which could be either “master” or “slave”. “Masters” are responsible for the cluster's health and stability. In large deployments with a lot of cluster nodes, it’s recommended to have more than one dedicated “master.” Typically, a dedicated “master” will not store data or create indexes. Thus, there should be no chance of being overloaded, by which the cluster health could be endangered.

“Slaves” are used as “workhorses” which can be loaded with data tasks. Even if a “slave” node is overloaded, the cluster health shouldn’t be affected seriously, provided there are other nodes to take additional load.

The setting which determines the role of the server is called node.master. If you have only one Elasticsearch node, you should leave this option commented out so that it keeps its default value of true — i.e. the sole node should be also a master. Alternatively, if you wish to configure the node as a slave, remove the # character at the beginning of the node.master line, and change the value to false:

/etc/elasticsearch/elasticsearch.yml

...
node.master: false
...

Another important configuration option is node.data, which determines whether a node will store data or not. In most cases, this option should be left to its default value (true), but there are two cases in which you might wish not to store data on a node. One is when the node is a dedicated “master,” as we have already mentioned. The other is when a node is used only for fetching data from nodes and aggregating results. In the latter case, the node will act up as a “search load balancer”.

Again, if you have only one Elasticsearch node, you should leave this setting commented out so that it keeps the default true value. Otherwise, to disable storing data locally, uncomment the following line and change the value to false:

/etc/elasticsearch/elasticsearch.yml

...
node.data: false
...

Two other important options are index.number_of_shards and index.number_of_replicas. The first determines into how many pieces (shards) the index will be split into. The second defines the number of replicas that will be distributed across the cluster. Having more shards improves the indexing performance while having more replicas makes searching faster.

Assuming that you are still exploring and testing Elasticsearch on a single node, it’s better to start with only one shard and no replicas. Thus, their values should be set to the following (make sure to remove the # at the beginning of the lines):

/etc/elasticsearch/elasticsearch.yml

...
index.number_of_shards: 1
index.number_of_replicas: 0
...

One final setting which you might be interested in changing is path.data, which determines the path where data is stored. The default path is /var/lib/elasticsearch. In a production environment, it’s recommended that you use a dedicated partition and mount point for storing Elasticsearch data. In the best case, this dedicated partition will be a separate storage media which will provide better performance and data isolation. You can specify a different path.data path by uncommenting the path.data line and changing its value:

/etc/elasticsearch/elasticsearch.yml

...
path.data: /media/different_media
...

Once you make all the changes, please save and exit the file. Now you can start Elasticsearch for the first time with the command:

sudo service elasticsearch start

Please allow at least 10 seconds for Elasticsearch to fully start before you are able to use it. Otherwise, you may get errors about not being able to connect.

Step 4 — Securing Elastic

Elasticsearch has no built-in security and can be controlled by anyone who can access the HTTP API. This section is not a comprehensive guide to securing Elasticsearch. Take whatever measures are necessary to prevent unauthorized access to it and the server/virtual machine on which it is running. Consider using iptables to further secure your system.

The first security tweak is to prevent public access. To remove public access edit the file elasticsearch.yml:

sudo nano /etc/elasticsearch/elasticsearch.yml
Find the line that contains the network.bind_host, uncomment it by removing the # character at the beginning of the line, and changing the value to localhost so it looks like this:

/etc/elasticsearch/elasticsearch.yml

...
network.bind_host: localhost
...

Warning: Because Elasticsearch doesn’t have any built-in security, it is very important that you do not set this to any IP address that is accessible to any servers that you do not control or trust. Do not bind Elasticsearch to a public or shared private network IP address!

Also, for additional security, you can disable dynamic scripts which are used to evaluate custom expressions. By crafting a custom malicious expression, an attacker might be able to compromise your environment.

To disable custom expressions, add the following line is at the end of the /etc/elasticsearch/elasticsearch.yml file:

/etc/elasticsearch/elasticsearch.yml

...
script.disable_dynamic: true
...

For the above changes to take effect, you will have to restart Elasticsearch with the command:

sudo service elasticsearch restart

Step 5 — Testing

By now, Elasticsearch should be running on port 9200. You can test it with curl, the command line client-side URL transfers tool, and a simple GET request like this:

curl -X GET 'http://localhost:9200'

You should see the following response:

Output of curl
{
  "status" : 200,
  "name" : "CentOS Node",
  "cluster_name" : "mysqluster",
  "version" : {
    "number" : "1.7.3",
    "build_hash" : "05d4530971ef0ea46d0f4fa6ee64dbc8df659682",
    "build_timestamp" : "2015-10-15T09:14:17Z",
    "build_snapshot" : false,
    "lucene_version" : "4.10.4"
  },
  "tagline" : "You Know, for Search"
}

If you see a response similar to the one above, Elasticsearch is working properly. If not, make sure that you have followed correctly the installation instructions and you have allowed some time for Elasticsearch to fully start.

Step 6 — Using Elasticsearch

To start using Elasticsearch, let’s add some data first. As already mentioned, Elasticsearch uses a RESTful API, which responds to the usual CRUD commands: Create, Read, Update, and Delete. For working with it, we’ll use again curl.

You can add your first entry with the command:

curl -X POST 'http://localhost:9200/tutorial/helloworld/1' -d '{ "message": "Hello World!" }'

You should see the following response:

Output
{"_index":"tutorial","_type":"helloworld","_id":"1","_version":1,"created":true}

With curl, we have sent an HTTP POST request to the Elasticseach server. The URI of the request was /tutorial/helloworld/1. It’s important to understand the parameters here:

the tutorial is the index of the data in Elasticsearch.

helloworld is the type.

1 is the id of our entry under the above index and type.

You can retrieve this first entry with an HTTP GET request like this:

curl -X GET 'http://localhost:9200/tutorial/helloworld/1'

The result should look like:

Output
{"_index":"tutorial","_type":"helloworld","_id":"1","_version":1,"found":true,"_source":{ "message": "Hello World!" }}

To modify an existing entry you can use an HTTP PUT request like this:

curl -X PUT 'localhost:9200/tutorial/helloworld/1?pretty' -d '
{
  "message": "Hello People!"
}'

Elasticsearch should acknowledge successful modification like this:

Output
{
  "_index" : "tutorial",
  "_type" : "helloworld",
  "_id" : "1",
  "_version" : 2,
  "created" : false
}

In the above example, we have modified the message of the first entry to “Hello People!”. With that, the version number has been automatically increased to 2.

You may have noticed the extra argument pretty in the above request. It enables a human-readable format so that you can write each data field on a new row. You can also “prettify” your results when retrieving data and get much nicer output like this:

curl -X GET 'http://localhost:9200/tutorial/helloworld/1?pretty'

Now the response will be in a much better format:

Output
{
  "_index" : "tutorial",
  "_type" : "helloworld",
  "_id" : "1",
  "_version" : 2,
  "found" : true,
  "_source":{ "message": "Hello World!" }
}

Setup web server to host ElasticSearch Application

These options depend on the webservers you have currently installed

For example, if you have installed nginx/varnish/apache you can use:
i. nginx -> custom port - This will proxy directly to ElasticSearch Application(RECOMMENDED)
ii. nginx/varnish/apache -> custom port - This will proxy to varnish cache and then to apache and then to ElasticSearch Application

So far we have added to and queried data in Elasticsearch. To learn about the other operations please check the API documentation.

Conclusion

That’s how easy it is to install, configure, and begin using Elasticsearch. Once you have played enough with manual queries, your next task will be to start using it from your applications.

Creating CWP iso image is easy, here are the steps:

- install and configure CWP
- power off the server
- create an image of the server

Since you had configured the server with IP, Hostname, and Email you will need to make that change after the image deployment.
You can run this command after starting the new server to change those settings:

Options:
--restart (Restart server after changes)
--newip (Set new IP in the server config)
--email (Set new admin email in the server config)
--hostname (Set new hostname in the server config)

sh /scripts/image_rebuild_server_config -r yes --newip 111.112.124.135 --email my@email.com --hostname vps1.domain.com

This script will do:
- change hostname
- change hostname in all services
- change hostname certificate
- change IP in all config
- change mysql password to random for database: postfix, root, roundcube
- clean log files
- download latest cwp version

https://hostname.domain.com:2083

https://hostname.domain.com:2087


https://100.110.120.130:2083


https://100.110.120.130:2087

Access to user panel over subdomain cpanel without port
cpanel.domain.com -> it's the same same as port 2083 but it is using user domain SSL certificate

if you need to access admin panel also without the port then you can edit nginx/apache hostname conf files and add config for subdomain, don't forget to add dns A record also.
/etc/nginx/conf.d/hostname.conf
/etc/nginx/conf.d/hostname-ssl.conf
/usr/local/apache/conf.d/hostname.conf
/usr/local/apache/conf.d/hostname-ssl.conf

webmail subdomain for access to Roundcube
webmail.domain.com -> using user domain SSL certificate

mail subdomain for access to the mail server
mail.domain.com -> STMP (including SSL/TLS per domain)
mail.domain.com -> IMAP/POP3 (including SSL/TLS per domain)

Auto-discover of mail configuration

Autologin
Single click login without the password
- From admin panel to user panel
- RoundCube (login to any email account)
- PHPMyAdmin (login as cpanel user)

User Panel
- Advanced File Manager

Migration tools

Restore account from cPanel (restores local cpmove file)
http://wiki.centos-webpanel.com/how-to-migrate-account-from-cpanel

cPanel to CWP server convert (removes cPanel and installs CWP but keeping all the accounts and databases)
http://wiki.centos-webpanel.com/cpanel-to-cwp-server-convert

cPanel Server Migration (migrates only accounts from the remote cpanel using SSH connection)
https://docs.control-webpanel.com/docs/admin-guide/user-accounts/cpanel-server-migration-new

3rd-Party Integrations
- Softaculous
- Sitepad
- WHMCS, Blesta, HostBill, ClientExec, Wisecp...
- Cloudlinux

If your main user panel login doesn't work for phpMyAdmin then please change the password in the user panel.

First install .net package repo from official Microsoft site :

rpm -Uvh https://packages.microsoft.com/config/rhel/7/packages-microsoft-prod.rpm

To install dotnet SDK 2.2

yum install dotnet-sdk-2.2 -y

To install dotnet SDK 3.1 LTS :

yum install dotnet-sdk-3.1 -y

you can check the main dotnet version by running this command :

dotnet --version

How to Create project - example:

to create a project or download any existing prebuild project you need to follow this steps:
Login into ssh with the username of your account user (ensure you enabled the full shell access for that user)
or from root you can run this command to change the user :

su - username -s /bin/bash

now go to user dir public_html:

cd /home/username/public_html

Download pre build ASP core project, we're using "movie-app-list" as an example :

git clone https://github.com/do-community/movie-app-list.git movie-app-list

Then enter the project dir :

cd movie-app-list

before we do the configuration this project needs MySQL DB and MySQL User for its data storage, you can simply create MySQL user and DB from user panel as well as admin panel.

after you create db and its user add those details in "appsettings.json"

{
  "Logging": {
    "LogLevel": {
      "Default": "Warning"
    }
  },
  "AllowedHosts": "*",
  "ConnectionStrings": {
    "MovieContext": "Server=localhost;User Id=user_netdb;Password=dqrESQo3;Database=user_netdb" } }

Now build the application :

you need to run this as root :

dotnet ef database update
dotnet build
dotnet publish

**upper commands will install csharp and all other dependencies.

Then fix the user permission :

cd /home/username/public_html
chown -R username:username *

Now again login to ssh as user and run the below command to start the application by default application is set to run on localhost:5000

Now start the application via this command :

$ su - username -s /bin/bash
$ /usr/bin/dotnet /home/username/public_html/movie-app-list/bin/Debug/netcoreapp2.2/publish/MvcMovie.dll

After that Ensure you've installed Nginx from CWP for port 5000 proxy go to :

ADMIN >> Webserver Settings >> Webserserver domain conf >> Choose user >> Create config under domain you need to proxy dotnet application >> Choose "nginx -> proxy -> (custom-port)" and choose force https template tick rebuild vhost and save changes and done.

* don't forget to set your app port!

go to the domain name and see your application is live there.

Troubleshoot guide :

If you get too many files open you simply need to increase the open file limit for the user from edit user account.

Step 1: Install Prerequisites:

yum groupinstall 'Development Tools' -y

Step 2: Install Yum Repository

NodeJS 12.x:

curl -sL https://rpm.nodesource.com/setup_12.x | bash -

NodeJS 11.x:

curl -sL https://rpm.nodesource.com/setup_11.x | bash -

NodeJS 10.x:

curl -sL https://rpm.nodesource.com/setup_10.x | bash -

NodeJS 8.x:

curl -sL https://rpm.nodesource.com/setup_8.x | bash -

Step 3: Install Nodejs and npm

yum install -y nodejs

Step 4: Confirm installation:
Run the below commands to check if nodejs is installed properly:

node -v
npm -v

The above command should output the version of nodejs and npm installed.For example, for version 12.X, you should see the something like below:

[root@cwp2 ~]# node -v
v12.6.0
[root@cwp2 ~]# npm -v
6.9.0

npm which is the package manager for nodejs can be used to search and install additional modules for nodejs.

npm search package
npm install package

Step 5: Setup webserver to host Nodejs Application
These options depend on the webservers you have currently installed

Example if you have installed nginx/varnish/apache you can use:
i. nginx -> custom port - This will proxy directly to nodejs Application(RECOMMENDED)
ii. nginx/varnish -> custom port - This will proxy to varnish cache and then to nodejs Application
iii. nginx/varnish/apache -> custom port - This will proxy to varnish cache and then to apache and then to nodejs Application

Configure webservers for domain to be used for nodejs Application:
CWP.Admin left Menu -> Webserver settings -> Webserver Domain Conf -> Select Username -> Select Domain/Subdomain -> create configuration -> select any option with (custom port) -> check the box – Rebuild webserver conf for domain on save -> Enter port number you have used in the application for the custom port, leave IP as 127.0.0.1 -> save changes

Step 6: Install AutoSSL
CWP.Admin left Menu -> Webserver settings -> SSL Certificates -> AutoSSL -> select user and domain -> install SSL

https://www.verisign.com/en_US/channel-resources/domain-registry-products/idn/idn-conversion-tool/index.xhtml

https://www.punycoder.com/

https://simpledns.com/idn-convert