RollingFileAppender class

addFilter

public void addFilter(Filter newFilter)
Add a filter to end of the filter list.

Specified by:
addFilter in interface Appender


위 filter에 log level을 설정할 수 있는데 검색하다 발견한 내용들.


 ALL    DEBUG   INFO    WARN    ERROR   FATAL   OFF
All                        
DEBUG  DEBUG                  
INFO   INFO   INFO               
WARN   WARN   WARN   WARN           
ERROR  ERROR  ERROR  ERROR  ERROR      
FATAL  FATAL  FATAL  FATAL  FATAL  FATAL  
OFF    OFF    OFF    OFF    OFF    OFF    OFF

위와 같은 모양으로 되어 있는 것으로 설정을 해봤다. 


LevelMinLevelMax를 All로 지정하면 모두가 나올 줄 알았지만 나오지 않았다.

LevelMin을 DEBUG, LevelMax를 ERROR로 테스트를 할 경우 위에  조건대로 로그가 생성되는 것을 확인하였다. 


로그가 생성이 안된다고 Log4Net이 문제라 생각하지말고 레벨 설정, 파일 경로 등을 설정해보라.


참고 : https://stackoverflow.com/questions/8926409/log4net-hierarchy-and-logging-levels

블로그 이미지

레몬도리 LemonDory

개발자의 이야기

출처 : http://www.mysqltutorial.org/mysql-error-handling-in-stored-procedures/


Summary: in this tutorial, you will learn how to use MySQL handler to handle exceptions or errors encountered in stored procedures.

When an error occurs inside a stored procedure, it is important to handle it appropriately, such as continuing or exiting the current code block’s execution, and issuing a meaningful error message.

MySQL provides an easy way to define handlers that handle from general conditions such as warnings or exceptions to specific conditions e.g., specific error codes.

Declaring a handler

To declare a handler, you use the  DECLARE HANDLER statement as follows:

If a condition whose value matches the  condition_value , MySQL will execute the statement and continue or exit the current code block based on the action .

The action accepts one of the following values:

  • CONTINUE :  the execution of the enclosing code block ( BEGIN … END ) continues.
  • EXIT : the execution of the enclosing code block, where the handler is declared, terminates.

The  condition_value specifies a particular condition or a class of conditions that activate the handler. The  condition_value accepts one of the following values:

  • A MySQL error code.
  • A standard SQLSTATE value. Or it can be an SQLWARNING , NOTFOUND or SQLEXCEPTION condition, which is shorthand for the class of SQLSTATE values. The NOTFOUND condition is used for a cursor or  SELECT INTO variable_list statement.
  • A named condition associated with either a MySQL error code or SQLSTATE value.

The statement could be a simple statement or a compound statement enclosing by the BEGIN and ENDkeywords.

MySQL error handling examples

Let’s look into several examples of declaring handlers.

The following handler means that if an error occurs, set the value of the  has_error variable to 1 and continue the execution.

The following is another handler which means that in case an error occurs, rollback the previous operation, issue an error message, and exit the current code block. If you declare it inside the BEGIN ENDblock of a stored procedure, it will terminate stored procedure immediately.

The following handler means that if there are no more rows to fetch, in case of a cursor or SELECT INTO statement, set the value of the  no_row_found variable to 1 and continue execution.

The following handler means that if a duplicate key error occurs, MySQL error 1062 is issued. It issues an error message and continues execution.

MySQL handler example in stored procedures

First, we create a new table named  article_tags for the demonstration:

The  article_tags table stores the relationships between articles and tags. Each article may have many tags and vice versa. For the sake of simplicity, we don’t create articles and tags tables, as well as the foreign keys in the  article_tags table.

Next, we create a stored procedure that inserts article id and tag id into the article_tags table:

Then, we add tag id 1, 2 and 3 for the article 1 by calling the insert_article_tags  stored procedure as follows:

After that, we try to insert a duplicate key to check if the handler is really invoked.

We got an error message. However, because we declared the handler as a CONTINUE handler, the stored procedure continued the execution. As the result, we got the tag count for the article as well.

MySQL Error Handling Example

If we change the CONTINUE in the handler declaration to EXIT , we will get an error message only.

Finally, we can try to add a duplicate key to see the effect.

MySQL error handling - duplicate keys

MySQL handler precedence

In case there are multiple handlers that are eligible for handling an error, MySQL will call the most specific handler to handle the error first.

An error always maps to one MySQL error code because in MySQL it is the most specific. An SQLSTATEmay map to many MySQL error codes, therefore, it is less specific. An SQLEXCPETION or an SQLWARNING is the shorthand for a class of SQLSTATES values so it is the most generic.

Based on the handler precedence’s rules,  MySQL error code handler, SQLSTATE handler and SQLEXCEPTIONtakes the first, second and third precedence.

Suppose we declare three handlers in the  insert_article_tags_3 stored procedure as follows:

We try to insert a duplicate key into the article_tags table by calling the stored procedure:

As you see the MySQL error code handler is called.

MySQL handler precedence

Using a named error condition

Let’s start with an error handler declaration.

What does the number 1051 really mean? Imagine you have a big stored procedure polluted with those numbers all over places; it will become a nightmare to maintain the code.

Fortunately, MySQL provides us with the DECLARE CONDITION statement that declares a named error condition, which associates with a condition.

The syntax of the DECLARE CONDITION statement is as follows:

The condition_value  can be a MySQL error code such as 1015 or a SQLSTATE value. The condition_valueis represented by the condition_name .

After the declaration, we can refer to condition_name instead of condition_value .

So we can rewrite the code above as follows:

This code is obviously more readable than the previous one.

Notice that the condition declaration must appear before handler or cursor declarations.

블로그 이미지

레몬도리 LemonDory

개발자의 이야기

출처: http://purumae.tistory.com/199


[MySQL / Stored Procedure] 에러 핸들링 시리즈


아래와 같은 테이블을 만들고, Stored Procedure 실행 중에 SQL Exception 이 발생했을 때, 디버깅에 필요한 단서를 저장합니다.

1
2
3
4
5
6
7
8
9
10
11
12
13
CREATE TABLE `error_log` (
    `error_log_id` SMALLINT(5UNSIGNED NOT NULL AUTO_INCREMENT COMMENT '에러 로그 ID',
    `proc_name` VARCHAR(100NOT NULL COMMENT '프로시저 이름',
    `proc_step` TINYINT(3UNSIGNED NOT NULL COMMENT '프로시저 내에서 에러가 발생한 스텝 번호',
    `sql_state` VARCHAR(5NOT NULL COMMENT 'SQLSTATE',
    `error_no` INT(11NOT NULL COMMENT '에러 번호',
    `error_msg` TEXT NOT NULL COMMENT '에러 메세지',
    `call_stack` TEXT NULL COMMENT '프로시저 호출 파라미터',
    `proc_call_date` DATETIME(0NOT NULL COMMENT '프로시저 호출 일자',
    `log_date` DATETIME(0NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '로그 적재 일자',
    PRIMARY KEY (`error_log_id`))
ENGINE = InnoDB
COMMENT = 'DB 런타임 에러 로그';
cs


샘플로 사용할 Stored Procedure의 일부

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
CREATE DEFINER=CURRENT_USER() PROCEDURE `usp_add_terms_agreement_for_campaign`(
      IN pi_int_campaign_id int UNSIGNED    -- 캠페인 ID
    , IN pi_int_social_user_id int UNSIGNED -- 소셜 유저 ID
    , IN pi_dt5_now datetime(0)             -- 현재 서버 시각
    , OUT po_int_return int                 -- 리턴 값
)
DETERMINISTIC
SQL SECURITY DEFINER
CONTAINS SQL
COMMENT '
author : doeyull.kim
e-mail : purumae@gmail.com
created date : 2018-05-03
description : 캠페인 약관에 동의합니다.
parameter : 
      IN pi_int_campaign_id int UNSIGNED    -- 캠페인 ID
    , IN pi_int_social_user_id int UNSIGNED -- 소셜 유저 ID
    , IN pi_dt5_now datetime(0)             -- 현재 서버 시각
    , OUT po_int_return int                 -- 리턴 값
return value :
    0 = 에러가 없습니다.
    -1 = 예상하지 않은 런 타임 오류가 발생하였습니다.
'
proc_body: BEGIN
    DECLARE v_vch_proc_name varchar(100DEFAULT 'usp_add_terms_agreement_for_campaign';
    DECLARE v_iny_proc_step tinyint UNSIGNED DEFAULT 0;
    DECLARE v_txt_call_stack text;
    DECLARE v_vch_sql_state varchar(5);
    DECLARE v_int_error_no int;
    DECLARE v_txt_error_msg text;
 
    DECLARE EXIT HANDLER FOR SQLEXCEPTION
    BEGIN
        GET DIAGNOSTICS CONDITION v_vch_sql_state = RETURNED_SQLSTATE
            , v_int_error_no = MYSQL_ERRNO
            , v_txt_error_msg = MESSAGE_TEXT;
 
        ROLLBACK;
 
        SET v_txt_call_stack = CONCAT('{"pi_int_campaign_id":'IF(pi_int_campaign_id IS NULL'null', pi_int_campaign_id)
            , ',"pi_int_social_user_id":'IF(pi_int_social_user_id IS NULL'null', pi_int_social_user_id)
            , ',"pi_dt5_now":'IF(pi_dt5_now IS NULL'null', CONCAT('"', pi_dt5_now, '"'))
            , '}'
        );
 
        INSERT error_log (proc_name, proc_step, sql_state, error_no, error_msg, call_stack, proc_call_date, log_date)
        VALUES (v_vch_proc_name, v_iny_proc_step, v_vch_sql_state, v_int_error_no, v_txt_error_msg, v_txt_call_stack, pi_dt5_now, NOW(0));
 
        SET po_int_return = -1;
 
        RESIGNAL;
    END;
cs


1. 변수 선언

25
26
27
28
29
30
    DECLARE v_vch_proc_name varchar(100DEFAULT 'usp_add_terms_agreement_for_campaign';
    DECLARE v_iny_proc_step tinyint UNSIGNED DEFAULT 0;
    DECLARE v_txt_call_stack text;
    DECLARE v_vch_sql_state varchar(5);
    DECLARE v_int_error_no int;
    DECLARE v_txt_error_msg text;
cs

v_vch_proc_name : 현재 Stored Procedure의 이름

v_vch_proc_step : SQL Exception이 발생한 구문위 위치를 찾기 위해 사용합니다. 번거롭고 원시적이지만 긴 Stored Procedure를 디버깅해야할 때 유용합니다.

v_txt_call_stack : input parameter 값을 json 형태로 변환하여 담습니다. (5.6과의 호환을 위해 json 유형으로 선언하지 않습니다.)

v_vch_sql_state v_int_error_no v_txt_error_msg : GET DIAGNOSTICS 구문으로 얻는 SQL STATE, Error Number, Error Message 를 담을 변수


2. DECLARE ... HANDLER 구문

32
33
.
.
52
    DECLARE EXIT HANDLER FOR SQLEXCEPTION
    BEGIN
 
 
    END;
cs

SQLEXCEPTION condition 이 되었을 때, BEGIN ~ END 블럭의 내용을 실행하고 EXIT 즉, Stored Procedure 실행을 강제로 종료합니다.


3. GET DIAGNOSTICS 구문

34
35
36
        GET DIAGNOSTICS CONDITION v_vch_sql_state = RETURNED_SQLSTATE
            , v_int_error_no = MYSQL_ERRNO
            , v_txt_error_msg = MESSAGE_TEXT;
cs

GET DIAGNOSTICS 구문으로 디버깅에 필요한 다음 세 가지 condition information 을 가져옵니다.

RETURNED_SQLSTATE
MYSQL_ERRNO
MESSAGE_TEXT


4. Call Stack 추출

40
41
42
43
44
        SET v_txt_call_stack = CONCAT('{"pi_int_campaign_id":'IF(pi_int_campaign_id IS NULL'null', pi_int_campaign_id)
            , ',"pi_int_social_user_id":'IF(pi_int_social_user_id IS NULL'null', pi_int_social_user_id)
            , ',"pi_dt5_now":'IF(pi_dt5_now IS NULL'null', CONCAT('"', pi_dt5_now, '"'))
            , '}'
        );
cs

Stored Procedure의 input parameter 값을 JSON 문자열로 만들어 v_txt_call_stack 변수에 담습니다.

TIP!!!
개발자가 매번 이 부분을 손으로 코딩하는 것은 바람직하지 않습니다. (단순 반복 작업 & 실수하기 쉬움) -> 자동화 하세요~


5. INSERT `error_log` ...

46
47
        INSERT error_log (proc_name, proc_step, sql_state, error_no, error_msg, call_stack, proc_call_date, log_date)
        VALUES (v_vch_proc_name, v_iny_proc_step, v_vch_sql_state, v_int_error_no, v_txt_error_msg, v_txt_call_stack, pi_dt5_now, NOW(0));
cs


6. 마무리

49
50
51
        SET po_int_return = -1;
 
        RESIGNAL;
cs

output parameter po_int_return

- Stored Procedure 가 정상적으로 실행 : SET po_int_return = 0;
- SQL Exception 이 발생하여 `error_log` 테이블에 로깅 : SET po_int_return = -1;


RESIGNAL

- error condition 정보를 Stored Procedure를 호출한 클라이언트에게 전합니다.
- RESIGNAL 하지 않으면 Stored Procedure를 호출한 클라이언트는 SQL Exception 이 발생했다는 사실을 감지하지 못하기 때문에, RESIGNAL 이 필요합니다.


블로그 이미지

레몬도리 LemonDory

개발자의 이야기

출처 : https://blog.fabioiotti.com/ip-address-batch/


This is a way to get the computer's IP address from a Batch script with a single line of code. Jump to TL;DR for the copy-paste ready code.

So... what do we need the IP address of this computer for? Do we need the our network address or our public address?

Getting Local IP Address

If the IP we need is the internal address (the one within current network's boundaries), we can ask our local DNS server (which knows our machine and our address). Local DNS server is running inside our router.

The correct way to proceed would be to run ipconfig command and parse it's output, then handpick from the list of results the address we actually want.

What if we want this procedure to be completely automatic? Then we have two options: ask the DNS server with nslookup command; or use local cache with pingcommand. What we will be doing is ping our own machine and see what Windows decided to be the best address to use.

We will use ping command to ping our local machine using IPv4 and sending only one packet. Since pinging the local machine, this operation should not halt and should never fail. When offline, this command will ping 127.0.0.1.

C:\Users\User>ping -4 -n 1 %ComputerName%  
Pinging MY-COMPUTER [192.168.1.139] with 32 bytes of data:  
Reply from 192.168.1.139: bytes=32 time<1ms TTL=128

Ping statistics for 192.168.1.139:  
    Packets: Sent = 1, Received = 1, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:  
    Minimum = 0ms, Maximum = 0ms, Average = 0ms

Let's first check what happens if our locale is not English...

C:\Users\User>ping -4 -n 1 %ComputerName%  
Esecuzione di Ping MY-COMPUTER [192.168.1.139] con 32 byte di dati:  
Risposta da 192.168.1.139: byte=32 durata<1ms TTL=128

Statistiche Ping per 192.168.1.139:  
    Pacchetti: Trasmessi = 1, Ricevuti = 1,
    Persi = 0 (0% persi),
Tempo approssimativo percorsi andata/ritorno in millisecondi:  
    Minimo = 0ms, Massimo =  0ms, Medio =  0ms

We can extract the line with the IP inside square brackets by piping the results into findstr command.

C:\Users\User>ping -4 -n 1 %ComputerName% | findstr "["  
Pinging MY-COMPUTER [192.168.1.139] with 32 bytes of data:  

And finally, we can discard anything outside of the brackets.

C:\Users\User>for /f "delims=[] tokens=2" %a in ('ping -4 -n 1 %ComputerName% ^| findstr [') do echo %a  
192.168.1.139  

Do we need this code to be Batch? Let's create a local_ip.bat file with the following contents…

@echo off

for /f "delims=[] tokens=2" %%a in ('ping -4 -n 1 %ComputerName% ^| findstr [') do set ThisIP=%%a

echo %ThisIP%  

And we are done. 😄

Note: all this commands are available since Windows XP and we never rely on localized strings, we can thus use this script anywhere.

Getting Public IP Address

If we need the public address of the outer machine in current network, then we need to ask for a little help from the outside: we have to call an external service and ask where they see our request coming from.

We well be using the free ipify service.

C:\Users\User>powershell Invoke-RestMethod api.ipify.org  
188.216.217.9  

Note: I also tried with nslookup myip.opendns.com. resolver1.opendns.com command but it didn't prove reliable as it failed in some of my networks.

Same as before: let's make this a Batch script. Name it public_ip.bat and paste the following code…

@echo off

for /f %%a in ('powershell Invoke-RestMethod api.ipify.org') do set ThisIP=%%a

echo %ThisIP%  

Note: unlike the script to read network IP address, this one relies on PowerShellwhich is only included since Windows 7. This script will thus not work on Windows XPand Vista out of the box.

Thanks for reading, I hope this guide will prove useful to you. Feel free to leave a comment in the section below! 😁


TL;DR

@echo off

for /f "delims=[] tokens=2" %%a in ('ping -4 -n 1 %ComputerName% ^| findstr [') do set NetworkIP=%%a

for /f %%a in ('powershell Invoke-RestMethod api.ipify.org') do set PublicIP=%%a

echo Network IP: %NetworkIP%  
echo Public IP: %PublicIP%  

References

https://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/nslookup.mspx?mfr=true

https://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/findstr.mspx?mfr=true

https://www.microsoft.com/resources/documentation/windows/xp/all/proddocs/en-us/ping.mspx?mfr=true

http://stackoverflow.com/q/5898763/1135019


블로그 이미지

레몬도리 LemonDory

개발자의 이야기

    "_id" : ObjectId("5bee629dee0fc9d332b2107b"), 

    "no" : NumberLong(30001), 

    "name" : "레몬과일가게", 

    "products" : [

        {

            "id" : NumberLong(50001), 

            "name" : "레몬", 

            "price" : NumberInt(4), 

        }, 

        {

            "id" : NumberLong(50002), 

            "name" : "사과", 

            "price" : NumberInt(12), 

        }, 

        {

            "id" : NumberLong(50003), 

            "name" : "파인애플", 

            "price" : NumberInt(3), 

        }, 

        {

            "id" : NumberLong(50004), 

            "name" : "귤", 

            "price" : NumberInt(3), 

        }

    ]

mongodb 에서   30001번 과일가게에서 레몬의 가격을 변경하기 위한 코드 샘플

var filter = Builders<store>.Filter.Where(c => c.no == 30001 && c.products.Any(e => e.id == 50001));

var update = Builders<player>.Update.Set(x => x.products[-1].price, 2000);

var option = new UpdateOptions() { IsUpsert = false }; 


출처 : https://stackoverflow.com/questions/31453681/mongo-update-array-element-net-driver-2-0


https://stackoverflow.com/questions/31277679/how-to-find-and-update-an-item-inside-of-a-array-of-an-object-in-an-array-acc

블로그 이미지

레몬도리 LemonDory

개발자의 이야기

출처 : https://stackify.com/log4net-guide-dotnet-logging/


If you have been doing .NET programming very long, odds are, you have touched a few projects that used log4net. It has been around for a long time and is very similar to log4j which is for Java. Log4net is a tool I have used forever and I wanted to share a tutorial to use for .Net logging!

What is log4net and why should you use it, or any C# logging framework?

I still find it hard to believe how many people I talk to that don’t use any sort of C# logging framework. Personally, it would be one of the first things I add to any new project.

Logging frameworks are important because they make it easy to write your logs to different places by simply changing your configuration. You can write your .NET logs to a file on disk, a database, a log management system or potentially dozens of other places, all without changing your code.

 

Getting Started: How to Install log4net via Nuget

1. Add log4net Package

Starting with log4net is as easy as installing a Nuget package. You can use the Visual Studio UI to search for it and install it, or just run this quick command from the Package Manager Console.

PM> Install-Package log4net

2. Add log4net.config file

Add a new file to your project in Visual Studio called log4net.config and be sure to set a property for the file. Set “Copy to Output Directory” to “Copy Always”. This is important because we need the log4net.config file to be copied to the bin folder when you build and run your app.

To get you started quickly, copy this log4net config and put it in your new log4net.config file. This will log messages to the console and a log file both. We will discuss more about logging appenders further down.

 <log4net>
    <root>
      <level value="ALL" />
      <appender-ref ref="console" />
      <appender-ref ref="file" />
    </root>
    <appender name="console" type="log4net.Appender.ConsoleAppender">
      <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="%date %level %logger - %message%newline" />
      </layout>
    </appender>
    <appender name="file" type="log4net.Appender.RollingFileAppender">
      <file value="myapp.log" />
      <appendToFile value="true" />
      <rollingStyle value="Size" />
      <maxSizeRollBackups value="5" />
      <maximumFileSize value="10MB" />
      <staticLogFileName value="true" />
      <layout type="log4net.Layout.PatternLayout">
        <conversionPattern value="%date [%thread] %level %logger - %message%newline" />
      </layout>
    </appender>
  </log4net>

3. Tell log4net to Load Your Config

The next thing we need to do is tell log4net where to load it’s configuratin from so that it actually works. I suggest putting this in your AssemblyInfo.cs file. You can find it under the Properties section in your project.

Add this to the bottom of your AssemblyInfo file.

[assembly: log4net.Config.XmlConfigurator(ConfigFile = "log4net.config")]

4. Log Something!

Now you can modify your app to log something and try it out!

    class Program
    {
        private static readonly log4net.ILog log = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);

        static void Main(string[] args)
        {
            log.Info("Hello logging world!");
            Console.WriteLine("Hit enter");
            Console.ReadLine();
        }
    }

Log Appenders: What They Are and Which Ones You Need to Know

Appenders are how you direct where you want your logs sent. The most popular of the standard appenders are most likely the RollingFileAppender and ConsoleAppender.

I would also try the DebugAppender if you want to see your log statements in the Visual Studio Debug window so you don’t have to open a log file.

If you are using a Console, checkout the ColoredConsoleAppender.

Colored console for log4net

COLORED CONSOLE FOR LOG4NET

 

Make Good Use of Multiple Log Levels and Filter by Them

Be sure to use Debug, Info, Warning, Error, and Fatal logging levels as appropriate within your code. Don’t lot everything as Debug. Be sure to think about how you would be viewing the logs and what you want to see it later when coding your logging statements.

You can specify in your log4net config which log4net logging levels you want to log. This is really valuable if you want to specify only certain levels to be logged to a specific log appender or to reduce logging in production. This also you to log more or less data without changing your code.

log4net levels:

  • All – Log everything
  • Debug
  • Info
  • Warn
  • Error
  • Fatal
  • Off – Don’t log anything

Advanced Topics & log4net Best Practices

1. Define Your LogManager Object as Static

Declaring any variable in your code has overhead. When I have been doing some profiling sessions in the past to optimize code, I have noticed that the constructors on the LogManager object can use a lot of CPU.

Declare it as static and use this little trick so you don’t have to hard code the class type.

   private static readonly log4net.ILog log 
       = log4net.LogManager.GetLogger(System.Reflection.MethodBase.GetCurrentMethod().DeclaringType);

2. How to Enable log4net’s Own Internal Debug Logging

From time to time, you may have problems with a specific appender, or issues working with it.

To help resolve these issues, enable internal log4net logging via your web.config file.

<configuration>
   <appSettings>
      <add key="log4net.Internal.Debug" value="true"/>
   </appSettings>
</configuration>

You can then specify where the logging is written to.

<configuration>
...

<system.diagnostics>
    <trace autoflush="true">
        <listeners>
            <add 
                name="textWriterTraceListener" 
                type="System.Diagnostics.TextWriterTraceListener" 
                initializeData="C:\tmp\log4net.txt" />
        </listeners>
    </trace>
</system.diagnostics>

...
</configuration>

3. Do Not Send Your Logs to a Database Table with the AdoAppender

Trying to query logs in SQL is very difficult if you log any real volume. You are much better off sending your logs to Elasticsearch or a log management service that can provide full-text indexing and more functionality with your logs.

 

4. Do Not Send Emails on Every Exception

The last thing you want to do is send any sort of emails from an appender. They either get ignored over time or something starts throwing a lot of exceptions and then your app starts sending thousands of errors. Although, there is an SmtpAppender if you really want to do this.

 

5. How to Send Alerts for Exceptions

If you want to send alerts about exceptions, send your exceptions to an error trackingproduct, like Retrace, which is designed for this. They can also de-dupe your errors so you can figure out when an error is truly new, track its history, and track error rates.

 

6. Send Your Logs to a Log Management System to View Them Across Servers

Capturing logs and logging them to a file on disk is great. But if you want to search your logs across multiple servers and applications, you need to send all of your logs to a central repository. There are a lot of log management solutions that can help you with this, or you can even setup your own elasticsearch cluster for it.

If you want to query all the files on disk, consider using VisualLogParser.

 

7. Use Filters to Suppress Certain Logging Statements

Filters can be configured to suppress specific log messages. Take a look at these examples.

Here’s how you can filter by the text on the log messages.

<filter type="log4net.Filter.StringMatchFilter">
  <stringToMatch value="test" /> <!-- Can filter by string or regex -->
</filter>

And here, you can filter by the log level.

<filter type="log4net.Filter.LevelRangeFilter">
   <levelMin value="INFO" />
   <levelMax value="FATAL" />
</filter>

8. You Can Make Your Own Custom log4net Appenders

If you want to do something that the standard appenders do not support, you can search online for one or write your own.

One example could be an appender for writing to Azure Storage. Once upon a time, we wrote one to send our logs to Azure Table storage to centralize them. We couldn’t really query them due to the lack of full-text indexing, but we could view them.

As an example of a custom appender, you can review the source code for our appender for sending logs to Retrace.

 

9. Customize Your Layout in Your Logs with log4net Pattern Layouts

You can modify your configuration to change what fields are outputting and in what format using pattern layouts.

    <appender name="LogFileAppender" type="log4net.Appender.RollingFileAppender">
      <param name="File" value="stackify.log" />
      <param name="AppendToFile" value="true" />
      <rollingStyle value="Size" />
      <maxSizeRollBackups value="10" />
      <maximumFileSize value="10MB" />
      <staticLogFileName value="true" />
      <layout type="log4net.Layout.PatternLayout">
        <param name="ConversionPattern" value="%-5p %d{MM-dd hh:mm:ss.ffff}  [%thread]  %m%n" />
      </layout>
    </appender>

Using the layout above, write the level (%p), current date time (%d), thread # (%thread), the message (%m) and a new line (%n). The “-5” in the “%-5p” is to set the width of the field to 5 characters.

Here are some other notable fields you can log, although they can have a big performance impact to your app and would not be recommended for high volume logging on a production application.

  • %method – name of the method where the log message was written
  • %stacktrace{level} – output a stack trace to show where the log message was written
  • %type – type of the caller issuing the log request. Mostly likely your class name
  • %line – the line number from where your logging statement was logged

A layout like this:

<layout type="log4net.Layout.PatternLayout">
        <param name="ConversionPattern" value="%-5p%d{ yyyy-MM-dd HH:mm:ss} – [%thread] %m method:%method %n stacktrace:%stacktrace{5} %n type:%type %n line: %line %n" />
</layout>

Produces a log message like this:

ERROR 2017-02-06 09:38:10 – [10] Error downloading web request method:ThrowWebException 
 stacktrace:Microsoft.VisualStudio.HostingProcess.HostProc.RunUsersAssembly > System.AppDomain.ExecuteAssembly > System.AppDomain._nExecuteAssembly > ConsoleApplication1.Program.Main > ConsoleApplication1.Class1.ThrowWebException 
 type:ConsoleApplication1.Class1 
 line: 26 

10. Use the Diagnostic Contexts to Log Additional Fields

You can also log custom fields to help give some more context about the user, customer or transaction related to the log statements.

The below example sets a custom property called “customer”. You can then modify your log4net pattern layout to include “%property{customer}” to output it in your logs.

            log4net.ThreadContext.Properties["customer"] = "My Customer Name";

            log.Debug("We are going to try and do a web request");

            try
            {
                Class1.ThrowWebException();
            }
            catch (Exception ex)
            {
                log.Error("Error trying to do something", ex);
            }
            log.Debug("We are done with the web request");

11. How to Correlate Log Messages by Web Request Transaction

Additionally, you can assign objects in contexts to use what it calls “active property values.” When the log message is written, the ToString() method will be called which can dynamically do something.

This can be used to write transaction IDs to help correlate messages to a web requestor transaction!

        //Create our little helper class
        public class ActivityIdHelper
        {
            public override string ToString()
            {
                if (Trace.CorrelationManager.ActivityId == Guid.Empty)
                {
                    Trace.CorrelationManager.ActivityId = Guid.NewGuid();
                }

                return Trace.CorrelationManager.ActivityId.ToString();
            }
        }

In your global.asax or Startup.cs class, subscribe to an event for when a request first starts.

        public override void Init()
        {
            base.Init();
            this.Error += WebApiApplication_Error;
            this.BeginRequest += WebApiApplication_BeginRequest;
            this.EndRequest += WebApiApplication_EndRequest;

        }

        void WebApiApplication_BeginRequest(object sender, EventArgs e)
        {
            //set the property to our new object
            log4net.LogicalThreadContext.Properties["activityid"] = new ActivityIdHelper();

            log.Debug("WebApiApplication_BeginRequest");
        }

If you add “%property{activity}” to your pattern layout, you can now see a transaction ID in your logs like so. Your log messages may still look like spaghetti, but at least you can easily see which ones go together.

DEBUG 02-06 02:51:58.6347 – a69640f7-d47d-4aa4-99c9-13cfd9ab93c2 WebApiApplication_BeginRequest 
DEBUG 02-06 02:51:58.6382 – a69640f7-d47d-4aa4-99c9-13cfd9ab93c2 Starting KitchenAsync - Call redis
DEBUG 02-06 02:51:58.9315 – b8a3bcee-e82e-4298-b27f-6481b256b5ad Finished KitchenAsync
DEBUG 02-06 02:51:59.1285 – a69640f7-d47d-4aa4-99c9-13cfd9ab93c2 Call Webclient
DEBUG 02-06 02:51:59.1686 – 54218fab-bd1b-4c77-9ff8-ebef838dfb82 WebApiApplication_BeginRequest
DEBUG 02-06 02:51:59.1746 – 54218fab-bd1b-4c77-9ff8-ebef838dfb82 Starting KitchenAsync - Call redis
DEBUG 02-06 02:51:59.4378 – a69640f7-d47d-4aa4-99c9-13cfd9ab93c2 Finished KitchenAsync
DEBUG 02-06 02:51:59.6450 – 54218fab-bd1b-4c77-9ff8-ebef838dfb82 Call Webclient
DEBUG 02-06 02:51:59.9294 – 54218fab-bd1b-4c77-9ff8-ebef838dfb82 Finished KitchenAsync

12. How to Log ASP.NET Request Details

You could use the same strategy as above to dynamically grab ASP.NET request info to add to your log message.

        public class WebRequestInfo
        {
            public override string ToString()
            {
                return HttpContext.Current?.Request?.RawUrl + ", " + HttpContext.Current?.Request?.UserAgent;
            }
        }

        void WebApiApplication_BeginRequest(object sender, EventArgs e)
        {
            log4net.LogicalThreadContext.Properties["activityid"] = new ActivityIdHelper();
            log4net.LogicalThreadContext.Properties["requestinfo"] = new WebRequestInfo();

            log.Debug("WebApiApplication_BeginRequest");
        }

13. How to Do Structured Logging, or Log an Object or Properties with a Message

By default, you can log an object to it and it will serialize it with its default renderers.

log.Debug(new {color="red", int1 = 1});

Output:

DEBUG 2017-02-06 15:07:25 – [8] { color = red, int1 = 1 }

But what if you want to log your entire log message as JSON?

There are several nuget packages related to log4net and JSON, but the support and docs for all of them seem a little sketchy.

I would recommend just making your own JsonLayout class that does it. There is a good sample on GitHub. You could then control exactly how you log the JSON and which fields you log.

Output from the GitHub JsonLayout:

{
	"processSessionId" : "225ba696-6607-4abc-95f6-df8e0438e898",
	"level" : "DEBUG",
	"messageObject" : "Finished KitchenAsync",
	"renderedMessage" : "Finished KitchenAsync",
	"timestampUtc" : "2017-02-06T21:20:07.5690494Z",
	"logger" : "WebApp2.Controllers.TestController",
	"thread" : "69",
	"exceptionObject" : null,
	"exceptionObjectString" : null,
	"userName" : "IIS APPPOOL\\WebApp2",
	"domain" : "/LM/W3SVC/1/ROOT/WebApp2-10-131308895921693643",
	"identity" : "",
	"location" : "WebApp2.Controllers.TestController+d__27.MoveNext(C:\\BitBucket\\PrefixTests\\WebApp2\\Controllers\\TestController.cs:477)",
	"pid" : 14428,
	"machineName" : "LAPTOP-1UJ70V4E",
	"workingSet" : 352481280,
	"osVersion" : "Microsoft Windows NT 10.0.14393.0",
	"is64bitOS" : true,
	"is64bitProcess" : true,
	"properties" : {
		"requestinfo" : {},
		"activityid" : {},
		"log4net:UserName" : "IIS APPPOOL\\WebApp2",
		"log4net:Identity" : "",
		"log4net:HostName" : "LAPTOP-1UJ70V4E"
	}
}

If you want to really get the value of structured logging, you will want to send your logs to a log management tool that can index all the fields and enable powerful searching and analytics capabilities.

Learn more here: What is structured logging and why developers need it.

 

14. How to View log4net C# Logs by ASP.NET Web Request

Log files can quickly become a spaghetti mess of log messages. Especially with web apps that have lots of AJAX requests going on that all do logging.

I highly recommend using Prefix, Stackify’s FREE .NET Profiler to view your logs per web request, along with SQL queries, HTTP calls and much more.

transaction trace annotated



블로그 이미지

레몬도리 LemonDory

개발자의 이야기

출처 : https://gist.github.com/denji/8359866


NGINX Tuning For Best Performance

For this configuration you can use web server you like, i decided, because i work mostly with it to use nginx.

Generally, properly configured nginx can handle up to 400K to 500K requests per second (clustered), most what i saw is 50K to 80K (non-clustered) requests per second and 30% CPU load, course, this was 2 x Intel Xeon with HyperThreading enabled, but it can work without problem on slower machines.

You must understand that this config is used in testing environment and not in production so you will need to find a way to implement most of those features best possible for your servers.

First, you will need to install nginx

yum install nginx
apt install nginx

Backup your original configs and you can start reconfigure your configs. You will need to open your nginx.conf at /etc/nginx/nginx.conf with your favorite editor.

# you must set worker processes based on your CPU cores, nginx does not benefit from setting more than that
worker_processes auto; #some last versions calculate it automatically

# number of file descriptors used for nginx
# the limit for the maximum FDs on the server is usually set by the OS.
# if you don't set FD's then OS settings will be used which is by default 2000
worker_rlimit_nofile 100000;

# only log critical errors
error_log /var/log/nginx/error.log crit;

# provides the configuration file context in which the directives that affect connection processing are specified.
events {
    # determines how much clients will be served per worker
    # max clients = worker_connections * worker_processes
    # max clients is also limited by the number of socket connections available on the system (~64k)
    worker_connections 4000;

    # optmized to serve many clients with each thread, essential for linux -- for testing environment
    use epoll;

    # accept as many connections as possible, may flood worker connections if set too low -- for testing environment
    multi_accept on;
}

# cache informations about FDs, frequently accessed files
# can boost performance, but you need to test those values
open_file_cache max=200000 inactive=20s; 
open_file_cache_valid 30s; 
open_file_cache_min_uses 2;
open_file_cache_errors on;

# to boost I/O on HDD we can disable access logs
access_log off;

# copies data between one FD and other from within the kernel
# faster then read() + write()
sendfile on;

# send headers in one peace, its better then sending them one by one 
tcp_nopush on;

# don't buffer data sent, good for small data bursts in real time
tcp_nodelay on;

# reduce the data that needs to be sent over network -- for testing environment
gzip on;
gzip_min_length 10240;
gzip_proxied expired no-cache no-store private auth;
gzip_types text/plain text/css text/xml text/javascript application/x-javascript application/json application/xml;
gzip_disable msie6;

# allow the server to close connection on non responding client, this will free up memory
reset_timedout_connection on;

# request timed out -- default 60
client_body_timeout 10;

# if client stop responding, free up memory -- default 60
send_timeout 2;

# server will close connection after this time -- default 75
keepalive_timeout 30;

# number of requests client can make over keep-alive -- for testing environment
keepalive_requests 100000;

Now you can save config and run bottom command

nginx -s reload
/etc/init.d/nginx start|restart

If you wish to test config first you can run

nginx -t
/etc/init.d/nginx configtest

Just For Security Reason

server_tokens off;

Nginx Simple DDoS Defense

This is far away from secure DDoS defense but can slow down some small DDoS. Those configs are also in test environment and you should do your values.

# limit the number of connections per single IP
limit_conn_zone $binary_remote_addr zone=conn_limit_per_ip:10m;

# limit the number of requests for a given session
limit_req_zone $binary_remote_addr zone=req_limit_per_ip:10m rate=5r/s;

# zone which we want to limit by upper values, we want limit whole server
server {
    limit_conn conn_limit_per_ip 10;
    limit_req zone=req_limit_per_ip burst=10 nodelay;
}

# if the request body size is more than the buffer size, then the entire (or partial)
# request body is written into a temporary file
client_body_buffer_size  128k;

# headerbuffer size for the request header from client -- for testing environment
client_header_buffer_size 3m;

# maximum number and size of buffers for large headers to read from client request
large_client_header_buffers 4 256k;

# read timeout for the request body from client -- for testing environment
client_body_timeout   3m;

# how long to wait for the client to send a request header -- for testing environment
client_header_timeout 3m;

Now you can do again test config

nginx -t # /etc/init.d/nginx configtest

And then reload or restart your nginx

nginx -s reload
/etc/init.d/nginx reload|restart

You can test this configuration with tsung and when you are satisfied with result you can hit Ctrl+C because it can run for hours.

DoS HTTP/1.1 and above: Range Requests

By default max_ranges is not limited. DoS attacks can many Range-Requests (Impact on stability I/O).

Socket Sharding in NGINX 1.9.1+ (DragonFly BSD and Linux 3.9+)

Socket typeLatency (ms)Latency stdev (ms)CPU Load
Default15.6526.590.3
accept_mutex off15.5926.4810
reuseport12.353.150.3

Thread Pools in NGINX Boost Performance 9x! (Linux)

Multi-threaded sending of files is currently supported only Linux. Without sendfile_max_chunk limit, one fast connection may seize the worker process entirely.

Happy Hacking!


블로그 이미지

레몬도리 LemonDory

개발자의 이야기

우분투 apt로 설치 시에 apt-key를 설정하고 설치해야 하는 경우가 생겼다.

바로 mono인데 문제는 방화벽이 다 막혀있고 proxy로 연결이 가능한 상태인데....


sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80

위 명령을 사용하는데 hkp://keyserver.ubuntu.com:80이 연결이 안된다.

80포트 조차 막혀있다니...


다행히 우회할 수 있는 방법이 프록시 서버가 있었다.

구글을 검색하다보니 

--keyserver-options http-proxy=http://localhost:3128 을 옵션을 이용하면 프록시 서버를 이용해 설치가 가능하단다. 


예제로 mono 설치 시 사용하는 apt-key 명령을 전체 남긴다.

sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --keyserver-options http-proxy=http://localhost:3128 --recv-keys 3FA7E0328081BFF6A14DA29AA6A19B38D3D831EF 


블로그 이미지

레몬도리 LemonDory

개발자의 이야기