A few days ago, the American Institute of industry and Applied Mathematics (SIAM) announced the list of members in 2020, and 28 outstanding scholars in the field of Applied Mathematics and computing science were elected. Among them, Professor Sun Defeng, a department friend of our department and professor of Hong Kong University of science and technology, was elected because of his outstanding contribution in the field of cone optimization, especially in the field of algorithm and software in matrix optimization.
About Professor Sun Defeng
Sun Defeng, Professor of optimization and operational research, Department of Applied Mathematics, Hong Kong University of technology. In 1989 and 1992, he received his bachelor's and master's degrees from the Department of mathematics, Nanjing University, and his doctor's degree from the Chinese Academy of Sciences in 1995. After that, he completed his post doctoral research at the University of New South Wales, Australia. In 2000, he began to teach in the Department of mathematics of National University of Singapore. In 2009, he was promoted to a full professor. In 2017, he joined Hong Kong Polytechnic University. Professor Sun delfeng mainly studies continuous optimization, and has made a series of important breakthrough achievements in the theory, algorithm and application of matrix optimization, especially the non smooth Newton method. He has written many large-scale and complex optimization software, such as the general large-scale semi positive planning software sdpnal / sdpnal +, the related matrix calibration program, and the latest applicable to a variety of Statistical regression model package lassonal. As a result, he was awarded the bile orchard Hays Award for international mathematical planning in 2018 every three years, in recognition of his outstanding contributions and collaborators in computational mathematical planning. At present, Professor Sun's research focuses on establishing the next generation method foundation for big data optimization and application.
He used to be the editor in chief of Asian Pacific J. operational research and the editorial board member of mathematical programming series B. now he is the editorial board member of Mathematical Programming Series A, Siam Journal on optimization, China Science Mathematics, Journal of the Operations Research Society of China, Journal of computational mathematics.
Professor Sun's award encouraged the students at school. Recently, we have the honor to invite Professor Sun for an exclusive interview. Professor Sun shared his experience and insights in mathematics research, and also made some valuable suggestions for the study and career of mathematics students.
Next, let's go into this exclusive interview.
PART 1
Academic research
Editor: first of all, I would like to congratulate the professor on his successful election as Siam fellow. We understand that as a chair professor of Applied Optimization and operational research in the Department of mathematics, Hong Kong University of technology, you have fruitful scientific research achievements. Can you briefly introduce your current research fields and achievements?
Professor Sun: I mainly do algorithm research. The most important content involved is optimization algorithm, which is used to solve some problems in real life, such as traffic, work assignment, linear planning and so on. My other research direction is large-scale optimization algorithm in machine learning, which belongs to big data science. The application fields of these algorithms are very broad, including machine learning, statistics, finance and so on.
Group photo of Professor Sun and Professor Terry rockafellar
Editor: your current research focuses on building the next generation method foundation for big data optimization and application. What is the main research content in this field? What kind of application does it have?
Professor Sun: there are many applications in this field, such as automatic driving. We need to make decisions quickly according to different routes and give feasible solutions. For example, various production arrangements of large companies, such as Alibaba, Tencent, Huawei, etc., also involve complex optimization problems, so we need to make optimization decisions every day. The common characteristic of these problems is that the data scale is large, and the solution time is very limited, so it needs to give a quick and efficient solution, which involves the data algorithm. If we use the traditional method to solve the problem, the dimension of the problem may be thousands of dimensions, that is, it will take thousands of years to use the supercomputer, which is not feasible, so we need to study the structure of the problem and its solutions, design efficient algorithms, and quickly get the results, which is the main problem of big data optimization and application of the next generation methods.
Professor Sun and mathematician Zhang Yitang
Editor: you have made a very important breakthrough in the non smooth Newton method. Can you introduce how you made a breakthrough in this area?
Prof. Sun: I was in non smooth analysis during my master's degree. We know that many functions are not differentiable in practical problems, such as absolute value functions, but we are more concerned about some of the relevant information they have, which leads to a new discipline -- nonsmooth analysis, which I was very interested in at that time. We know that the famous Newton method is used to solve the nonlinear equation, that is, to approach the solution of the equation through successive linearization, but the premise of this method is that the corresponding function of the equation must be continuous and differentiable, so how to deal with the non differentiable equation? At that time, some scholars began to study the generalized Newton method to solve nonsmooth equations. We soon found that the nonsmooth functions in the nonsmooth equations are mostly semi smooth, which is worse than smooth, but much better than continuous functions, so as to ensure the fast convergence of the generalized Newton method. This is very important in thought, because before we thought that Newton method must have smoothness. At the same time, we notice that there is a kind of nonsmooth function with sparsity, that is to say, it can perfectly describe the sparsity of the solution of the problem. When we find this function, we will bring it into the equation and construct the equivalent equation of the problem. The equation may be non differentiable or semi differentiable, but it has sparsity, so we can simplify the solution of the problem, which is the main idea of semi smooth sparse Newton method.
For example, the dimension of the solution of the original problem is 10 million, but now we can find the component information of the solution which plays an important role in it automatically by using the semi smooth sparse Newton method, maybe only dozens or thousands of them play a role, and then solve these problems with low dimension, which greatly simplifies the complexity of the operation. But if we do not use the nonsmooth sparse Newton method, we may have to solve 10 million dimensional problems.
I began to study nonsmooth function in 80-90's, find out that special function, or construct it by myself, and then apply nonsmooth Newton method to solve it. But at that time, the computer technology was underdeveloped, the practicability of this method was not well reflected, and the mainstream method at that time was smooth algorithm. Smooth method is very stable, but it often turns sparse problem into non sparse problem, which is not a big problem when the dimension is small. Until later entering the era of big data, semi smooth sparse Newton method showed its importance. Through this method, some big data problems of operation research optimization can be solved perfectly, and the solution of these problems is almost impossible before.
PART2
Study experience
Editor: what are your impressive experiences during your study in NTU? How do you think the cultivation of NTU will affect your current research?
Professor Sun: many of the teachers I met when I was studying in Nantah were just returned from overseas visits. They brought a lot of advanced knowledge in the world. They basically used famous classic professional books as teaching materials and taught with great dedication. In particular, the non smooth optimization seminar has a profound impact on me. The non smooth analysis I have learned plays a key role in solving the large-scale optimization problems that I am studying now. It's no exaggeration to say that my research work in optimization algorithm and non smooth analysis theory in the next 30 years is basically benefited from the solid foundation laid during my study in Nantah. Now I think that the curriculum arrangement of Nantah in terms of optimization at that time was very scientific, which was very advanced in China and first-class in the world.
Group photo of teachers and students in summer of 1992
Another thing that impresses me is that Mr. He xuchu, the founder of Nantah optimization, stipulated that the graduate students of Computational Mathematics in Nantah must be able to write programs. This was a vision in the era when computers were still scarce! I am now able to optimize software development without fear, and I really benefit from this requirement of Mr. He.
In the past few years, I have read many books on optimization according to my own interests, many of which are still instructive. Therefore, my starting point is relatively good, and I was very lucky to meet the wave of upsurge of big data scientific development later, which enabled me to apply mathematical theory knowledge to big data.
A group photo of Professor Sun in the summer of 1992
Editor: we see that you also have a long time of overseas study experience. Can you share your experience of studying abroad?
Professor Sun: after my master's degree, I continued to study for a Ph.D. in the Institute of Applied Mathematics, Chinese Academy of Sciences. After that, I went to Australia to complete post doctoral training, and contacted more new information. I also had extensive exchanges with some top experts in the world. In 2000, I applied to the National University of Singapore, where I met a lot of internationally renowned scholars and carried out some theoretical research on Optimization stability, which basically laid the foundation for my research in the field of algorithm.
2007 National University of Singapore distinguished scientist Award
Then I began to return to the field of computation, mainly to explore the use of nonsmooth equations to solve large-scale convex programming problems, which took nearly 20 years. These results also won the belle orchard Hays Award for computing optimization in 2018, and were recognized by international peers for this exploration.
Photo and medal of the belle orchard Hays Award
Here's an interesting story: I learned a course called matrix computing in Nanjing University, and I felt that the relevant theories were very beautiful. After that, although I chose the optimization direction during my master's degree, I always thought about matrix. Later, when I was doing optimization research in Singapore, I felt that it was a pity not to study the matrix theory, so I started to develop a new subject called matrix optimization. It would be a dream that I wanted to explore matrix computing at the beginning to combine the two.
I remember that when I first went abroad, I was trembling. I wondered if my achievements in isolation could stand the comparison of others? Later, it was found that the foundation of mathematical optimization in NTU and Academy of Sciences at that time was very solid, the results could stand the comparison of international peers, some places would have their own unique features, and they were particularly comfortable in communicating with international experts, which is my experience of living abroad.
Professor Sun arrived in Sydney at the end of July 95 after graduation
PART3
Experience sharing
Editor: what are your experience and experience in scientific research? How do you think mathematics students should improve their research ability?
Professor Sun: my biggest feeling is to do what I like, to invest more in what I'm interested in, and to be curious. There was no special reason for the non smooth analysis in that year, but I thought it was very interesting, so I insisted on doing it. I remember when I was reading sophomore year, I saw Professor Sheng Songbai from the second floor holding a document with "Newton method" in his hand“